A teacher of mine told us, "Don't try to imagine 4D space. You can't. If you think you can imagine a 4D hypersphere, you're wrong. It's just not possible."
And everytime someone talks about how to imagine higher-dimensional space, I have to think back, and I'm convinced that my teacher was right. Our intuitive understanding of 2D and 3D just doesn't generalise to higher dimensions.
A hypersphere is nothing like a sphere, just like a sphere is nothing like a circle, which is again nothing like a line segment.
If you talk about multidimensional geometries, stick to the math, and don't try imagining it. Analogies to 2D or 3D objects (it's smooth, but like a spike) are pointless and don't lead to new insights. We really need to stick to precise, mathematical language if we want to work with higher dimensional geometry.
I think he was wrong. It's pretty hard but not impossible. You don't visualize them directly as 3D objects though, you visualize 3D objects with metadata attached everywhere. While you navigate the space in your mind's eye you have an extra layer of information that flows with you.
Consider Alicia Boole Stott. She created cardboard models of the various cross-sections of the 6 regular polytopes. Including the 24-cell which has no equivalent in 3D, the 120-cell and the 600-cell which are super complex figures. I think if you read her story you'll also be convinced that she indeed visualized the figures. [1]
I spent quite a lot of time earlier this year on this visualization exercise on the 4 simplest regular polytopes while working on an toy application to manipulate 4D objects and tesseractic honeycomb in VR.
You can have a good grasp of where every edge go and the angle between every faces. I could count the number of vertices and edges of a 16-cell or 24-cell just by navigating around the figure in my head. The tesseract and 16-cell are good starting points.
However there is a 4-dimensional sphere (3-sphere) that you can easily access: the possible rotations of any object in 3-dimensional space is topologically and metrically isomorphic to a (half of a) 3-sphere, in the same way as the rotations of an object in 2-dimensional space are isomorphic to a 1-sphere (circle).
Think of an object’s current orientation as the “north pole” of our abstract 4-dimensional sphere. Rotating the object around any axis pushes you toward the equator, which you reach once you have rotated the object halfway around (the equator is made from all 180° turns, and is topologically a 2-sphere). Continuing to rotate the object takes you towards the “south pole”, which is where you get if you rotate your object 360°. To get back to the north pole, rotate by 360° again.
One other way you can get at the 3-sphere is by taking a stereographic projection into all of 3-dimensional space (plus a point at infinity). If you have some 3-dimensional shapes drawn on the surface of your 3-sphere, these will get distorted under the stereographic projection, but locally angles will be preserved, just like a stereographic projection of the 2-sphere onto a piece of paper.
I think that your terminology "3-sphere" (which is standard) as synonymous with "4-dimensional sphere" (which is not) may be confusing. Of course the 3-sphere is 3-dimensional, hence the name; it is just that it is canonically embedded in 4-dimensional Euclidean space (and not in 3-dimensional, flat Euclidean space).
Fair enough. Hopefully nobody was confused. I was trying to meet the grandparent poster halfway. His comment was “[...] Don't try to imagine 4D space. You can't. If you think you can imagine a 4D hypersphere [...]”
I've been thinking about the 4D visualization problem and I am becoming increasingly convinced we can develop some cool tools to help wrap our mind around it using VR goggles.
There are two ways to draw a 3D object in 2D. One is to draw slices, or projections. The other is to use tricks of "perspective" (points of infinity and shading) It is very hard to visualize a 3D object simply from the slices model.
When we try to draw 4D objects in 3D, we often simply construct models that are slices of the 4D objects and then try to extrapolate back in our minds what the 4D object "looks like". This is even more difficult to wrap our brains around.
I postulate that we can develop similar tricks of perspective and shading that will help use see a 4D object if we draw on a 3D canvas. I don't think we can do so on a 2D canvas and make any sense of it. However, I'm hopeful that using VR goggles and drawing in 3D might be the trick.
I've only really just started thinking about this idea. I have a ton of books on constructing perspective for 3D objects on 2D surfaces. Much has been written about it since the Renaissance. It is only in the last couple of years that we've had an effective and accessible 3D canvas to work with. (constructing hundreds of sculptures is not really practical)
I don't think a lot of work has been done with VR goggles yet for 4D visualizations.
I find some value from picturing 4D objects as 3D objects moving and changing size and or shape through space. In the ball example. At T:1/4 and T:3/4 I see the same 3D ball shape with different balls. At time 0, 0.5, and 1 they are gone.
As such it's natural for a 4D where to fit in the middle as it can be centered at T:1/2 and you just need to find the intersection point in 4D space where the balls touch.
Now, this does not work for rotations, but it's better than nothing.
>A teacher of mine told us, "Don't try to imagine 4D space. You can't. If you think you can imagine a 4D hypersphere, you're wrong. It's just not possible."
Your brain is a computer. If a computer can manipulate 4D hypersphere, so can you.
You can't visualize it, but that's a different thing. You can't visualize a 3D sphere either, only 2D projection.
I'm not sure I agree you can't visualize a 3d object. You can't draw one, I'll give you that, but we spend all our time in 3d space. Saying the brain's representation of that is 2D seems dubious.
Not the brain, the eye's representation is 2D. The brain's can be 3D, despite the brain not receiving any 3D input, so I don't see why the brain can't represent 4D as well.
High dimensions are indeed very, what shall I say, non-intuitive. Bellman introduced, or perhaps popularized, the term 'curse of dimensionality'. High dimensions may also comes with its blessing, 'concentration of measure' a phenomenon where randomness concentrates into events of certainty in high dimensions. I checked out their Wikipedia pages, while the former has a lucid description, the page on the latter is quite technical.
Both these notions play a significant role in topics like machine learning, statistics, function approximations etc. Its sometimes a delight sometimes sheer frustration to see these two phenomena play it out.
One other mindblowing fact is that if you project the mass of the sphere onto a line that goes through the origin, you get a Gaussian distribution as the dimension goes to infinity.
If you're interested in more technical details about the weird geometry of balls, I can't recommend enough the first few pages of "An Elementary Introduction to Modern Convex Geometry" [1]. Ironically, the author's last name is Ball.
It should be clarified that this fact refers to a ball of volume 1 in N dimensions. Since the volume is held constant, as the dimension goes to infinity, so does the radius.
To visualize, take a hyper-cube of length 1. It's volume is 1 unit for any n-dimensional hyper-cube. Scribe a hyper-sphere touching all the sides. How about its volume with increasing dimension? That decreases! (with respect to the cube). To visualize, think about the distance of the cube vertex from the centre, it increases with increasing dimension.
In other words, Most of the cube's volume is closer to its vertex, not in the ball in the centre!
You can start by working out the ratio of volumes between a sphere of radius 1 and the enclosing cube in 1, 2, and 3 dimensions (i.e. a line segment, a circle, and a sphere, respectively). You'll see that the ratio starts at 1 and decreases as the dimension increases.
> You can start by working out the ratio of volumes between a sphere of radius 1 and the enclosing cube in 1, 2, and 3 dimensions (i.e. a line segment, a circle, and a sphere, respectively). You'll see that the ratio starts at 1 and decreases as the dimension increases.
Be careful using low-dimensional phenomena as a guide to the asymptotic behaviour! If you take a sphere of radius 2, rather than radius 1, then the analogous ratio increases for a while (until n = 5), but then decreases to 0.
That's doubtful, as no random variables are apparently involved in the questions of geometry. The situation with the Gauss function is perhaps similar to that of pi, which appeared originally in geometry of circles and only later was discovered in formulae for probability distributions or sums of infinite series. Neither Gauss function nor pi seem to be naturally connected to questions of probability; but applications of the probability theory to special situations lead to expressions that use these most basic mathematical concepts.
Another counterintuitive result in this vein is to think about randomly sampling uniformly distributed points in a sphere. As the dimension rises, the chance of finding a sample within, say, r/100 of the surface goes to infinity. In high dimensions, all of the mass of the sphere is concentrated at the edge.
Exactly! Now consider that deep networks that classify images are tasked with getting reliable statistics in very high dimensional spaces. Conv nets are being forced to map from high dimensional spaces down to a very low dimensional categorical decision. This is why weird classification errors you see in "adversarial examples" keep popping up. The learned classification boundaries are very spiky, and it's easy for the world to fall in between the spikes. More data can't solve it (at least not practically) because there are way too many gaps between the spikes.
A more tractable approach is to learn to use dynamics for perception rather than ("just") statistics. The dynamical physics of a ball rolling is much simpler (lower dimensional, more tractable) than a statistical view of millions of differently illuminated pixels hitting a camera.
Yes, but in direct contrast to Chomsky (who would say there's not enough data/time for kids to learn from) I am saying that there is a ton of rich dynamical data in the world around us all the time. Plenty to learn from.
Just plug a webcam into an adequate system and allow it to learn dynamics by trying to predict what it will see next.
Chomsky is almost right for the wrong problem: there won't ever be enough human-labeled data for good generalization. ;)
I think what Chomsky was saying in that particular debate was that purely statistical methods do not lead to true understanding, as opposed to more phenomenological theory.
That seems very intuitive if you think about the difference between a line segment and a circle. In 1d space a segment that extended S1: from 0 to 1 covers half the space of one that extends from S2: 0 to 2.
In x,y space you have that same 50:50 ratio at y=0. But, for every y > 0 the ratio decreases as a higher percentage of S1 is used up just reaching that y height than S2, until all of it is used up and the top segments are 100% s2.
In x,y,z space you get that same picture at z=0. But at any z > 0 more of that S1 is used to than the S2 segment so the ratio must be lower than in x,y or x space.
Moving to 4D and up the same simple idea just continues.
I've heard this phrased as "objects in high dimensions are mostly surface", which makes clear that it's not that there's a less dense "core" but that every interior point is near the surface.
That makes sense now that you mention it. The equation for the surface area of an n-sphere goes up as r^n, and the volume growth at any given radius is equal to the surface area at that radius.
I am not sure I would interpret this observation as spheres that are getting more spikey. Maybe what gets more spikey is the actual box, so the sphere has it easier to touch or even extend outside of it.
> I am not sure I would interpret this observation as spheres that are getting more spikey. Maybe what gets more spikey is the actual box, so the sphere has it easier to touch or even extend outside of it.
I think that this is not correct. In n dimensions, the cross section of a unit n-cube near the edge is a unit (n - 1)-cube, just as your intuition tells you. However, the cross section of a unit n-sphere near its intersection with the unit n-cube is a tiny, tiny (n - 1)-sphere (the radius of a cross section at height z = 1 − 𝜀 goes to 0 as n goes to ∞); this is the 'spikiness' that the author is discussing.
On the other hand, it seems to me (maybe I am wrong) that the solid n-dimensional angle of each corner of the n-cube is progressively smaller (as a ratio of full n-dimensional angle). So in that sense, each n-cube gets more spikey, and there is "less space" in each corner for each n-sphere that enclose the central n-sphere.
For a more rigorous treatment (and many other gems on the border of highly practical and highly theoretical), see Richard Hamming's great book "The Art of Doing Science and Engineering", available for download from Bret Victor's website:
- because (1-ε)^n/1^n goes to zero if n goes up, most of the volume of a cube gets concentrated near its surface.
- to go from a n-dimensional cube to a n-dimensional sphere, you have to cut away most of its surface (everything except for 2n 'caps' near the points with coordinates (0,0,0,...,0,0,±½,0,0,0,...,0,0,0)), and hence, most of its volume.
To see some concrete examples of how this sort of phenomenon causes surprising failures in deep networks classifying images, see "Intriguing properties of neural networks"[1]
Very interesting! I've done a lot of work with higher-dimensional spaces in NLP and Machine Learning (creating feature vectors with tens of thousands of dimensions, and then calculating cosines between vectors to estimate document similarity), but these ideas about spikey geometry are new to me and very insightful. Thanks!
So one visualisation of a sphere in very high dimensions is not something smooth and round, but something that is somehow simultaneously very symmetrical, and yet also very spikey.
So if for some reason you are trying to trying to make a loose analogy to the spikiness of hyperspheres to an older military mariner who has never seen a hedgehog, you could clarify by saying "you know, like the Hertz horns on a sea mine?"
And everytime someone talks about how to imagine higher-dimensional space, I have to think back, and I'm convinced that my teacher was right. Our intuitive understanding of 2D and 3D just doesn't generalise to higher dimensions.
A hypersphere is nothing like a sphere, just like a sphere is nothing like a circle, which is again nothing like a line segment.
If you talk about multidimensional geometries, stick to the math, and don't try imagining it. Analogies to 2D or 3D objects (it's smooth, but like a spike) are pointless and don't lead to new insights. We really need to stick to precise, mathematical language if we want to work with higher dimensional geometry.