ENTERTAINING THE FUTURE

Vol.33 No.3 Aug. 1999

---

ACM SIGGRAPH

Armchair Resolution (Solving the Image Size Problem from the Comfort and Safety of Your Own Living Room)

Mike Milne
FrameStore

August 99 Columns

Previously in this column I have examined some aspects of vision; I wondered about the ubiquity of the Windows paradigm, and how it might be introducing more problems than it solves; I also mused on the difficulties faced by digital artists attempting to depict visible light sources. It is another aspect of vision that I want to explore now: that of image size, and the allied subject of image resolution, and how it affects our entertainment.

The catalyst for these thoughts was a quick detour to the Tate Gallery to take in the new Jackson Pollock show, which I imagine will have been seen across the Atlantic earlier this year. (I found the early work interesting, late stuff disappointing - probably because the artist developed the fear that “anyone can splash out a Pollock,” and started to try to inject some content into his work. I would like to have seen more of my favourite works, those giant canvases that everyone recognizes as Pollocks immediately - precisely because they have been splashed out, I guess).

It’s a sobering thought that, when Jackson Pollock was my age, he’d already been dead for 10 years. However, this wasn’t the thought that came to mind as I admired the largest of the middle period pictures - I can’t remember what it was called, and quite frankly it doesn’t really matter - it was the thought that without the huge scale of the picture (about 16 feet wide, I think) it wouldn’t engage the attention in quite the way it does.

Basically, Pollock understood the psychology of screen size.

Size Does Matter

The human brain accords significance to the scenes presented to it through the visual system in a number of ways, not all of them fully understood (or even guessed at) by scientists. However, it seems fairly clear that one of the very first ways in which the environment is analysed is by subjective size - that is, the angle subtended at the eye by an object or scene. Of course the story doesn’t end there - if you were to enter a room and be faced by a six-foot high pile of dirty laundry on one side, and a small but deadly coral snake on the other, there is no doubt that the brain would assess the relative importance of the two images very quickly, and conclude that however large the pile of laundry, it was unlikely to slither towards you and administer a fatal bite to the ankle.

Other things being equal, however, size does matter - though not actual size, just apparent size. This is quite logical when you think about it - if an insect, however small, subtends a large angle at the eye, then it’s near enough to bite. Conversely, if a boulder - however large - appears small, that’s because it’s far away, and cannot harm you in the near future. (At least, that would have been the case until the invention of the mangonel in the 4th century BC, as enemies of Philip II of Macedonia would have discovered just moments before their untimely demise!)

Early Sight Decisions

Actually I suspect that the relationship between apparent size and relative importance was established in quite a crude and primitive way in the very early development of sighted organisms. The first eyes were no more than cells that were slightly more light sensitive than those in the surrounding tissue, and could just about distinguish bright sunlight from pitch dark. An organism possessing a number of these handy tools spread around its periphery could determine which direction to move by “counting votes” and moving with the majority - away from, or towards, the light, depending on the type of creature it was. Later developments favoured an arrangement in which the light-sensitive cells were arrayed in a cup-shaped depression in the surface, which cuts down the angle from which light could trigger the photocells, and so produces more accurate direction-finding. There would still be a basic, underlying connection between the proportion of cells firing and the intensity of motivation felt by the organism - and, probably, there would be some sort of threshold at about the 50 percent mark, since that’s the point at which the vote would be won by one faction or the other. (Now I know that this may seem extremely arbitrary and far-fetched, and hardly scientific - not even an experiment to back it up - but please bear with me. Who knows, I might even dream up a practical test or two, just to keep you awake!)

Of course the sophistication of our human optical systems, and the processing power behind them, is staggering when compared to those primitive forms of sight - but, I would suggest, that original response is still at the root of our reaction to the apparent size of the things we see. Deeply buried in the recesses of what some people call the “reptilian brain,” that ancient relic of our distant ancestors that provides our most innate and irresistible drives, there is a simple directive that accords priority to anything with large relative size.

Size Equals Importance?

And it’s worth noting that this apparently primitive mechanism has given useful guidance for many millions of years - right up to this century, in fact. Even after the emergence of human culture, perceived size was still in direct relation to physical importance. A vast pyramid spoke volumes about the importance of its creator, in times when buildings seldom rose above three or four stories. Medieval church leaders understood the principle well when they built the huge cathedrals that still invoke a feeling of awe when gazing up at the lofty pillars and enormous vaulted ceilings - the buildings’ designers deliberately exploited our innate reaction to size, in order to induce the feeling of reverence.

Things fall apart now that we’re in the age of symbols, when sizes can be deceptive. Size is no longer directly related to importance - consider the relative merits of a cashier’s cheque for a million dollars and a large sheet of brown wrapping paper; or a 40 foot roadside billboard for Ed’s Diner and a letter of dismissal from your employer; a diamond ring and a hula-hoop; a microchip and a haystack. Within our culture, we now can see the limitations of a size-related scale of importance for visual processing, but unfortunately the slow machinery of natural selection is blind to changes that happen over the course of a few thousand years - a mere eyeblink in evolutionary terms. Whether we like it or not, we’re victims of the tyranny of relative size.

So it is that, even though we are quite capable of distinguishing, on an intellectual level, between items in our visual field that are important to us and items that are not - irrespective of their apparent size - our inherent mental circuitry will still refuse to add that frisson of awe unless the required area of retinal cells is occupied; conversely, it will stubbornly insist on assigning total attention to any talentless goon who occupies the 60-foot screen at your local movie theatre, even when you know that this particular specimen of humanity has severe difficulty pronouncing words of more than one syllable.

And that’s just the effect that Jackson Pollock was exploiting when he painted those huge canvases. Let’s face it, they simply wouldn’t have the same impact if they were just a few inches across. In fact, they’d look suspiciously like a paper napkin that someone had splashed some paint on. Pollock went some way towards proving that, provided your canvas is big enough, you can splash out whatever you damn well please on it, and it still looks cool. Don’t get me wrong, though - I actually love those Pollocks, and if I had a few spare millions I’d like to own one. It’s just that, like all great artists, he used the tools at his disposal to maximum effect, and a large canvas was one of them.

Need for Bigger Displays

And where is all this leading? Well, I suppose it brings me to the inevitable conclusion that I need a bigger display system if I really want to enjoy watching TV. That’s why I find something essentially dissatisfying about watching a movie, or a drama, on a domestic television set. We’re all accustomed to it, and we can get totally absorbed in what we see on a small screen, but I maintain that there’s something missing . I’ll be honest - I would like to be better entertained, and I won’t be until I can harness some of that ol’ retinal magic that comes from big screen.

OK, you might say, there’re plenty of giant wide-screen TV sets down at the local electrical shop - and if they’re not large enough, you can get a projection system. The trouble is, whatever system you get, it’s not big enough - the angular size (that is, the apparent size, given by angle that the screen subtends at your eye) is still fairly small. “Well, sit closer to the screen, you dummy!” might be your response. And that’s where we hit the problem of image resolution - which we’ll come back to in just a couple of paragraphs. Meanwhile, we need to find out what size a screen should be if I’m going to take full advantage of my evolutionary past.

Let’s Do Some Tests

Now it’s time for a few practical experiments with the help of a handy tape measure. Aha! Now we’re doing real science! Right now, I’m looking at the screen of my iMac, which is about one foot wide, and my nose is... let’s see... almost exactly two feet from the screen (I’m a little long-sighted, as it happens). Some simple trigonometry (well, a calculator, actually) reveals that the angular size of the screen is about 28 degrees horizontally. Making a quick trip to the living room, and sitting in my favourite TV-watching chair, I find that the distance to the screen is nine feet, and the width of the screen is 1.5 feet. This equates to an angular size of 10 degrees. Even if I had one of those swanky new wide-screen plasma panel sets, which are about five feet across, I wouldn’t be seeing as large a picture as my little iMac. If I wanted to watch TV at the same angular size, I’d need a set that was nearly seven feet wide.

(You may wonder why, in metricated Europe, I’m still talking feet and inches. Well, I come from the generation whose childhood straddled the change in Britain from Imperial to Metric, so I’m bimetric, as one might say. However, since I’m from that generation, I’m of an age where my sight is - how shall I put it - not what it once was. Quite simply, those pesky millimeters are just too damn small to see. So now I do science in feet, and my smallest unit is about two or three inches - which I recognise as being one of the blurry black blobs about a quarter of the way between two blurry red blobs on the tape measure, and which I legitimise with the scientifically-correct-sounding label of 0.25 feet. Don’t worry - it’s plenty accurate enough for our purposes!)

At work today I did a quick survey of some of the animators’ viewing habits. They all use similar monitors, SGI 21-inchers which have a width of one foot and some blurry bits (sorry, that’s 1.25 feet. This is science, after all). The average viewing distance is just short of two feet, so the angular size of an SGI is about 34 degrees, or more than three times that of my domestic TV at normal viewing distance. Now why can’t I sit nearer to my home TV, and get that same angular size? Well, because at a closer distance even someone whose sight is as ... er... not quite what it was... as me, begins to distinguish the individual pixels that make up the picture. And, interestingly enough, the distance I sit from the screen is directly related to the angular size of the pixels. Now to you and me, who make our livings by shovelling bucketsful of these pixels around every hour, this is nursery school stuff. But even so, have you considered just what the angular size of pixels is, and how this translates to larger domestic screen sizes?

OK, time for another precision experiment with the blurry black and red blobs on my tape measure. This time, I’ve drawn some thick vertical black lines, in pairs, on a white piece of paper which I’ve taped to the side of our render farm rack. I walk backwards along the corridor, unreeling the tape measure as I go, and I note the distance at which the pairs of lines appear to become a single line. I repeat this for a number of different line separations, and calculate the angular size at each disappearing act. Interestingly, they’re fairly constant at 0.014 degrees, which means that I can distinguish about 72 lines per degree. I’m told that some time before metrication, when I had 20-20 vision, I would have been able to distinguish about 120 lines per degree. Ah well. So it goes.

What this little experiment tells me is that, if my TV screen is 10 degrees across, I should be able to resolve about 720 lines in that space. Now the pixel resolution of PAL television is...well, would you believe it? 720 pixels! So, that’s why I stay nine feet away. Any closer, and I’d see the dots. Actually, there’s reason to believe that we are prepared to put up with a pixel size larger than our smallest resolvable dot, presumably because our visual processing circuitry can lessen the effect.

Now we have to do just one more experiment. For this, I ask for a trusty assistant. I stand in front of a blank wall at a measured distance - a few feet - and stare at a fixed spot on the wall exactly in front of me. My willing helper, tape measure in hand, moves along the wall away from me, until I can’t see him clearly. He records the distance from the fixed spot, and this allows us to calculate the angular size of effective human vision, which is (in my case) about 120 degrees. There is indistinct vision and awareness of motion right out to about 160 degrees, but we’ll just concentrate on the smaller figure.

If you recall, I suggested that we need to occupy more than half the visual field to make full use of the old ancestral circuitry; this means that we should consider a screen of about 65 or 70 degrees angular size. As you’d expect, this is roughly what you’ll find if you take your tape measure down to the your local movie theatre and measure the screen size and the distance to your favourite seat. To get this sort of angular size in my living room, I’d need a screen that was about 10 feet across. At the current angular image resolution that I enjoy, that would translate to a resolution of about 5000 pixels - nearer 8,500 for keener-eyed youngsters. Of course this resolution is independent of actual screen size, and is determined by the angular size alone. This means that 8.5k pixels is an absolute limit, that we need never increase, if we’re happy with 70-degree screens.

Processing Power

So, finally, we come to the nitty-gritty. What sort of processing power will I need to manipulate this sort of image, in my ideal home cinema? And what sort of machine will people in my line of work (computer animators, I mean) be using to provide images for my giant screen? Assuming that 16:9 will become the universal screen ratio, then my 8.5K picture will be about 4.8K lines vertically, and will hold down about 41 million pixels and weigh in at about 86 megabytes per frame. This is roughly 100 times more than an uncompressed TV picture today, so we need an improvement of roughly two orders of magnitude over the current technology.

Using the old yardstick of the 18-month doubling, that give us a period of about 10 years before the technology catches up with my dream. And the person doing my job in 10 years (oh no, it won’t be me - I’ll be relaxing in a hammock on the shores of the Mediterranean, ice cubes clinking in the glass... dream on...) will be using a machine that runs at 45 gigahertz, with around 50 gigabytes of RAM, and a disc storage facility of about 10 terabytes. The images that come out of that machine will need a transfer speed of 17 gigabits per second, and a two-hour movie will need a storage space (assuming a compression ratio of, say, 6 to 1) of 15 thousand terabytes.

Now that’s what I call entertainment!

Mike Milne is Director of Computer Animation at FrameStore, which together with its sister company CFC, forms one of Europe’s largest digital effects teams. Mike recently completed Walking with Dinosaurs, a six-part British Broadcasting Corporation series, which begins airing in the fall of 1999.