A Biography of the Pixel

An interesting and fascinating look at the element used to hold a digital sample of an image: the pixel. Often misunderstood to mean the picture elements you see on your screen, a pixel is actually an element that is a digital representation (sample) of an analogue image. The book goes through how an analogue signal (sound, images, moving images) is transformed into a digital sample and then used to recreate an analogue output, followed by the history of films and animated films, the creation of computers and the creation of digital images and finally the dream of the author and the others, to create The Movie, a fully computer generated film that would finally appear in the form of Pixar's "Toy Story".

Here's a chapter by chapter look at the book

• Fourier’s Frequencies: The Music of the World: An introduction to Fourier waves is given, along with a history of its creator, Joseph Fourier, including his interactions with Napoleon and other French scientists at the time. The author uses Fourier waves to show how to determine the frequencies that make up sound waves. Then Fourier waves are show to use to describe complex images.

• Kotelnikov’s Samples: Something from Nothing: a history of Vladimir Kotelnikov is given, asserted by the author to have invented the Sampling Theorem, along with interactions with various Soviet people like Stalin. The author shows that when the Sampling Theorem is used with Fourier waves, it can convert analogue values into discrete value samples. Then by using an appropriate 'shaping wave' the digital samples can be used to accurately recreate the analogue values. The author also gives the distinction between a pixel (samples of analogue values) and a picture element (used to display a pixel on a display). He also shows how the misuse of Sampling Theory leads to artefacts in audio and images.

• Turing’s Computations: Eleventy-Eleven Skydillion: this chapter is on Alan Turing and his idea of computing using a device that follows simple rules. This simple device is Turing's Universal computer. Based on Turing's idea, von Neumann would find a way for the universal computer to be created (a computer architecture). The author describes how to program a computer and shows that a computer does it work via the manipulation of symbols, not numbers.

• Dawn of Digital Light: The Quickening: a history of the early British and American computer efforts is covered here. As it turns out, an early use of CRTs (cathode ray tubes) is as a form of computer memory. This would lead to the CRT being used as a way to display images based on data in memory (for example, displaying a graph based on values in memory, rather than showing the values directly). Early forms of computer animation and possibly visual computer games are also shown in this chapter.

• Movies and Animation: Sampling Time: here, the author shows how the Fourier transform and sampling theorem should work for movies: this is based on the speed of movement, details of film projection and on our persistence of vision and perception of motion. A History of film projectors is given, along with the beginnings of film animation. The difference between live and animated film is given and show to be time: live films are shown in real time, while animations can compress or stretch time in films. The art of making animated characters come to live via exaggerations is also shown. Coming back to the Sampling Theorem, the author shows how current films speeds were developed before sampling theorem known, and that current films depend more on the perception of motion by us than accurate sampling to represent movement. This is a reason why sampling artefacts (like wheels spinning backwards) exists in films.

• Shapes of Things to Come: this chapter shows the beginnings of computer graphics, which are representation of lines and surfaces of pictures in computers. The 'spline' line and triangle are shown as the basic elements used to create lines and surfaces that connect discrete points. The difference between CAD and picture based graphics is shown. Next, a history of the first 'epoch' in computer graphics is given, with ideas generated to render smooth lines and surfaces between shapes, to display computer models in 2D and 3D (in perspective) and the ability of computers to interactively modify the computer model in real time. These would set the stage for computer animation.

• Shades of Meaning: Moore's law makes an appearance, leading to an explosion of the ability of computers to display better computer graphics and to animate graphics. 3D shapes are now being rendered, as lots of triangles. Ideas in how to hide 3D surfaces, adding a light source and brightness, doing texture mapping and shading appear. Ironically, the first applications of computer graphics to aid in the production of 2D animation would run into problems. Unlike 3D, 2D computer graphics had no concept of objects, making animation of items that hiding and reappear (like an arm moving back and forth and being hidden by a body) difficult for a computer to handle. At this time, initial work on rendering short 3D animations would also be done.

• The Millennium and The Movie: this chapter would become more personal as it involves the author's personal experience. The ambitions to make The Movie by him and others would being here. Moore's Law was creating computers capable of more sophisticated and better ways of creating images and animating them. Besides generating scenery images, the desire to create animated characters that look alive take root: this is Character animation. Interestingly, the ability of computers to add blurring to images would lead to the ability of computers to represent realistic movement in computer animation. But the wait now is for good enough computation, as determined by Moore's Law. In the meantime, there was the creation of various animation and graphics departments that would lead to various groups like Pixar and others. Eventually, this would lead to the production of The Movie: "Toy Story".

• Finale: The Great Digital Convergence - this closing chapter summarizes the earlier chapters of the book. It also looks to the future of computer graphics in Virtual, Augmented and Mixed Reality. Other aspects include the manipulation of feature of actors. A section also looks at the ability of AI to modify images in ways not known then.