ENTERTAINING THE FUTURE
Vol.33 No.3 Aug. 1999
Armchair Resolution (Solving the Image Size Problem from the Comfort and Safety of Your Own Living Room)
Previously in this column I have examined some aspects of vision; I wondered about the ubiquity of the Windows paradigm, and how it might be introducing more problems than it solves; I also mused on the difficulties faced by digital artists attempting to depict visible light sources. It is another aspect of vision that I want to explore now: that of image size, and the allied subject of image resolution, and how it affects our entertainment.
The catalyst for these thoughts was a quick detour to the Tate Gallery to take in the new Jackson Pollock show, which I imagine will have been seen across the Atlantic earlier this year. (I found the early work interesting, late stuff disappointing - probably because the artist developed the fear that ďanyone can splash out a Pollock,Ē and started to try to inject some content into his work. I would like to have seen more of my favourite works, those giant canvases that everyone recognizes as Pollocks immediately - precisely because they have been splashed out, I guess).
Itís a sobering thought that, when Jackson Pollock was my age, heíd already been dead for 10 years. However, this wasnít the thought that came to mind as I admired the largest of the middle period pictures - I canít remember what it was called, and quite frankly it doesnít really matter - it was the thought that without the huge scale of the picture (about 16 feet wide, I think) it wouldnít engage the attention in quite the way it does.
Basically, Pollock understood the psychology of screen size.
Size Does Matter
The human brain accords significance to the scenes presented to it through the visual system in a number of ways, not all of them fully understood (or even guessed at) by scientists. However, it seems fairly clear that one of the very first ways in which the environment is analysed is by subjective size - that is, the angle subtended at the eye by an object or scene. Of course the story doesnít end there - if you were to enter a room and be faced by a six-foot high pile of dirty laundry on one side, and a small but deadly coral snake on the other, there is no doubt that the brain would assess the relative importance of the two images very quickly, and conclude that however large the pile of laundry, it was unlikely to slither towards you and administer a fatal bite to the ankle.
Other things being equal, however, size does matter - though not actual size, just apparent size. This is quite logical when you think about it - if an insect, however small, subtends a large angle at the eye, then itís near enough to bite. Conversely, if a boulder - however large - appears small, thatís because itís far away, and cannot harm you in the near future. (At least, that would have been the case until the invention of the mangonel in the 4th century BC, as enemies of Philip II of Macedonia would have discovered just moments before their untimely demise!)
Early Sight Decisions
Actually I suspect that the relationship between apparent size and relative importance was established in quite a crude and primitive way in the very early development of sighted organisms. The first eyes were no more than cells that were slightly more light sensitive than those in the surrounding tissue, and could just about distinguish bright sunlight from pitch dark. An organism possessing a number of these handy tools spread around its periphery could determine which direction to move by ďcounting votesĒ and moving with the majority - away from, or towards, the light, depending on the type of creature it was. Later developments favoured an arrangement in which the light-sensitive cells were arrayed in a cup-shaped depression in the surface, which cuts down the angle from which light could trigger the photocells, and so produces more accurate direction-finding. There would still be a basic, underlying connection between the proportion of cells firing and the intensity of motivation felt by the organism - and, probably, there would be some sort of threshold at about the 50 percent mark, since thatís the point at which the vote would be won by one faction or the other. (Now I know that this may seem extremely arbitrary and far-fetched, and hardly scientific - not even an experiment to back it up - but please bear with me. Who knows, I might even dream up a practical test or two, just to keep you awake!)
Of course the sophistication of our human optical systems, and the processing power behind them, is staggering when compared to those primitive forms of sight - but, I would suggest, that original response is still at the root of our reaction to the apparent size of the things we see. Deeply buried in the recesses of what some people call the ďreptilian brain,Ē that ancient relic of our distant ancestors that provides our most innate and irresistible drives, there is a simple directive that accords priority to anything with large relative size.
Size Equals Importance?
And itís worth noting that this apparently primitive mechanism has given useful guidance for many millions of years - right up to this century, in fact. Even after the emergence of human culture, perceived size was still in direct relation to physical importance. A vast pyramid spoke volumes about the importance of its creator, in times when buildings seldom rose above three or four stories. Medieval church leaders understood the principle well when they built the huge cathedrals that still invoke a feeling of awe when gazing up at the lofty pillars and enormous vaulted ceilings - the buildingsí designers deliberately exploited our innate reaction to size, in order to induce the feeling of reverence.
Things fall apart now that weíre in the age of symbols, when sizes can be deceptive. Size is no longer directly related to importance - consider the relative merits of a cashierís cheque for a million dollars and a large sheet of brown wrapping paper; or a 40 foot roadside billboard for Edís Diner and a letter of dismissal from your employer; a diamond ring and a hula-hoop; a microchip and a haystack. Within our culture, we now can see the limitations of a size-related scale of importance for visual processing, but unfortunately the slow machinery of natural selection is blind to changes that happen over the course of a few thousand years - a mere eyeblink in evolutionary terms. Whether we like it or not, weíre victims of the tyranny of relative size.
So it is that, even though we are quite capable of distinguishing, on an intellectual level, between items in our visual field that are important to us and items that are not - irrespective of their apparent size - our inherent mental circuitry will still refuse to add that frisson of awe unless the required area of retinal cells is occupied; conversely, it will stubbornly insist on assigning total attention to any talentless goon who occupies the 60-foot screen at your local movie theatre, even when you know that this particular specimen of humanity has severe difficulty pronouncing words of more than one syllable.
And thatís just the effect that Jackson Pollock was exploiting when he painted those huge canvases. Letís face it, they simply wouldnít have the same impact if they were just a few inches across. In fact, theyíd look suspiciously like a paper napkin that someone had splashed some paint on. Pollock went some way towards proving that, provided your canvas is big enough, you can splash out whatever you damn well please on it, and it still looks cool. Donít get me wrong, though - I actually love those Pollocks, and if I had a few spare millions Iíd like to own one. Itís just that, like all great artists, he used the tools at his disposal to maximum effect, and a large canvas was one of them.
Need for Bigger Displays
And where is all this leading? Well, I suppose it brings me to the inevitable conclusion that I need a bigger display system if I really want to enjoy watching TV. Thatís why I find something essentially dissatisfying about watching a movie, or a drama, on a domestic television set. Weíre all accustomed to it, and we can get totally absorbed in what we see on a small screen, but I maintain that thereís something missing . Iíll be honest - I would like to be better entertained, and I wonít be until I can harness some of that olí retinal magic that comes from big screen.
OK, you might say, thereíre plenty of giant wide-screen TV sets down at the local electrical shop - and if theyíre not large enough, you can get a projection system. The trouble is, whatever system you get, itís not big enough - the angular size (that is, the apparent size, given by angle that the screen subtends at your eye) is still fairly small. ďWell, sit closer to the screen, you dummy!Ē might be your response. And thatís where we hit the problem of image resolution - which weíll come back to in just a couple of paragraphs. Meanwhile, we need to find out what size a screen should be if Iím going to take full advantage of my evolutionary past.
Letís Do Some Tests
Now itís time for a few practical experiments with the help of a handy tape measure. Aha! Now weíre doing real science! Right now, Iím looking at the screen of my iMac, which is about one foot wide, and my nose is... letís see... almost exactly two feet from the screen (Iím a little long-sighted, as it happens). Some simple trigonometry (well, a calculator, actually) reveals that the angular size of the screen is about 28 degrees horizontally. Making a quick trip to the living room, and sitting in my favourite TV-watching chair, I find that the distance to the screen is nine feet, and the width of the screen is 1.5 feet. This equates to an angular size of 10 degrees. Even if I had one of those swanky new wide-screen plasma panel sets, which are about five feet across, I wouldnít be seeing as large a picture as my little iMac. If I wanted to watch TV at the same angular size, Iíd need a set that was nearly seven feet wide.
(You may wonder why, in metricated Europe, Iím still talking feet and inches. Well, I come from the generation whose childhood straddled the change in Britain from Imperial to Metric, so Iím bimetric, as one might say. However, since Iím from that generation, Iím of an age where my sight is - how shall I put it - not what it once was. Quite simply, those pesky millimeters are just too damn small to see. So now I do science in feet, and my smallest unit is about two or three inches - which I recognise as being one of the blurry black blobs about a quarter of the way between two blurry red blobs on the tape measure, and which I legitimise with the scientifically-correct-sounding label of 0.25 feet. Donít worry - itís plenty accurate enough for our purposes!)
At work today I did a quick survey of some of the animatorsí viewing habits. They all use similar monitors, SGI 21-inchers which have a width of one foot and some blurry bits (sorry, thatís 1.25 feet. This is science, after all). The average viewing distance is just short of two feet, so the angular size of an SGI is about 34 degrees, or more than three times that of my domestic TV at normal viewing distance. Now why canít I sit nearer to my home TV, and get that same angular size? Well, because at a closer distance even someone whose sight is as ... er... not quite what it was... as me, begins to distinguish the individual pixels that make up the picture. And, interestingly enough, the distance I sit from the screen is directly related to the angular size of the pixels. Now to you and me, who make our livings by shovelling bucketsful of these pixels around every hour, this is nursery school stuff. But even so, have you considered just what the angular size of pixels is, and how this translates to larger domestic screen sizes?
OK, time for another precision experiment with the blurry black and red blobs on my tape measure. This time, Iíve drawn some thick vertical black lines, in pairs, on a white piece of paper which Iíve taped to the side of our render farm rack. I walk backwards along the corridor, unreeling the tape measure as I go, and I note the distance at which the pairs of lines appear to become a single line. I repeat this for a number of different line separations, and calculate the angular size at each disappearing act. Interestingly, theyíre fairly constant at 0.014 degrees, which means that I can distinguish about 72 lines per degree. Iím told that some time before metrication, when I had 20-20 vision, I would have been able to distinguish about 120 lines per degree. Ah well. So it goes.
What this little experiment tells me is that, if my TV screen is 10 degrees across, I should be able to resolve about 720 lines in that space. Now the pixel resolution of PAL television is...well, would you believe it? 720 pixels! So, thatís why I stay nine feet away. Any closer, and Iíd see the dots. Actually, thereís reason to believe that we are prepared to put up with a pixel size larger than our smallest resolvable dot, presumably because our visual processing circuitry can lessen the effect.
Now we have to do just one more experiment. For this, I ask for a trusty assistant. I stand in front of a blank wall at a measured distance - a few feet - and stare at a fixed spot on the wall exactly in front of me. My willing helper, tape measure in hand, moves along the wall away from me, until I canít see him clearly. He records the distance from the fixed spot, and this allows us to calculate the angular size of effective human vision, which is (in my case) about 120 degrees. There is indistinct vision and awareness of motion right out to about 160 degrees, but weíll just concentrate on the smaller figure.
If you recall, I suggested that we need to occupy more than half the visual field to make full use of the old ancestral circuitry; this means that we should consider a screen of about 65 or 70 degrees angular size. As youíd expect, this is roughly what youíll find if you take your tape measure down to the your local movie theatre and measure the screen size and the distance to your favourite seat. To get this sort of angular size in my living room, Iíd need a screen that was about 10 feet across. At the current angular image resolution that I enjoy, that would translate to a resolution of about 5000 pixels - nearer 8,500 for keener-eyed youngsters. Of course this resolution is independent of actual screen size, and is determined by the angular size alone. This means that 8.5k pixels is an absolute limit, that we need never increase, if weíre happy with 70-degree screens.
So, finally, we come to the nitty-gritty. What sort of processing power will I need to manipulate this sort of image, in my ideal home cinema? And what sort of machine will people in my line of work (computer animators, I mean) be using to provide images for my giant screen? Assuming that 16:9 will become the universal screen ratio, then my 8.5K picture will be about 4.8K lines vertically, and will hold down about 41 million pixels and weigh in at about 86 megabytes per frame. This is roughly 100 times more than an uncompressed TV picture today, so we need an improvement of roughly two orders of magnitude over the current technology.
Using the old yardstick of the 18-month doubling, that give us a period of about 10 years before the technology catches up with my dream. And the person doing my job in 10 years (oh no, it wonít be me - Iíll be relaxing in a hammock on the shores of the Mediterranean, ice cubes clinking in the glass... dream on...) will be using a machine that runs at 45 gigahertz, with around 50 gigabytes of RAM, and a disc storage facility of about 10 terabytes. The images that come out of that machine will need a transfer speed of 17 gigabits per second, and a two-hour movie will need a storage space (assuming a compression ratio of, say, 6 to 1) of 15 thousand terabytes.
Now thatís what I call entertainment!
Mike Milne is Director of Computer Animation at FrameStore, which together with its sister company CFC, forms one of Europeís largest digital effects teams. Mike recently completed†Walking with Dinosaurs, a six-part British Broadcasting Corporation series, which begins airing in the fall of 1999.
9 Nole Street
London W1V 4AL
The copyright of articles and images printed remains with the author unless otherwise indicated.