Tuesday, April 10, 2018

Expanded gamut - when an idea's time has come

I enjoy researching the history of innovation.

Last week's post was the history of the creation of the board game Monopoly. It was a tale of innovation and deception; about good old capitalism at its best and at its worst. Today I will track the development of another innovation, but there is no deception and no villain in today's story. There is no moralistic message, just a practical message for would-be innovators.

So, mix up a Printer's Delight cocktail, and enjoy today's history lesson. For those not familiar with it, this drink is a combination of blue Curacao, Malbec, Yuengling, and Kahlua. The proportions are adjusted to match the appropriate color.

Try all 1,617 variations with the IT8 sampler!

The past few years, there has been a lot of hoopla about expanded gamut printing, which is to say, printing with inks beyond the standard four colors (cyan, magenta, yellow, and black). To the casual observer, one might conclude that this was a relatively recent idea. This is true... well... if one is willing to define recent in a more geologic sense.

Hallmark Cards

The earliest instance that I have found of expanded gamut printing dates all the way back to 1960 with a company called Hallmark Cards. If you are are not familiar with Hallmark, they invented the idea that you had to feel guilty about not sending a card to your mother on Mother's Day. I haven't found the original patent for the guilt thing, but it must have been filed somewhere around 1920.

Hallmark's unique printing problem is illustrated below. Greeting cards have a lot of pastel colors. These colors are often outside of the normal printing gamut. It is not always appreciated, but CMYK just can't get you highlights that are both very bright and highly saturated.

Karl Guyler (of Hallmark) said that Hallmark was in production with a six color process by 1962. The inks included fluorescent pink, yellow, and magenta. The system was so cool, that they had to give it a cool name that they trademarked: BigBox ColorTM. Although (as I mentioned before) Hallmark patented the guilt associated with their gilt, for some reason they did not patent this particular method of expanded gamut printing.

Why did this innovation happen? Hallmark had special niche needs that were not met by the existing technology.

Why didn't this innovation spread like wildfire? At the time, these were niche needs, and others didn't see a benefit.

Shoichi Shimada

The earliest patent I found on expanded gamut printing was from Shoichi Shimada of the Japanese company Dianippon Screen, filed in 1968. This patent describes a method where printing plates "are produced for reproducing color images with inks other than the standard inks …" They use three primary inks (cyan, magenta, and yellow), three secondary inks (for example, orange, green, and violet), along with black ink.

The diagram below shows in simplistic terms how the color separation is to work. Color space is divided into six pie slices. Actually, since color is three-dimensional, think of these as three apple slices. Each slice of the apple is assigned one of the process colors (CMY), one of the extra colors (in this case OGV), and black. The system determines which slice of apple the given color belongs in, and uses the appropriate inks for that apple slice to create the given color. 

Shimada-sam's color separation strategy

Shimada-san's patent mentions that printing of calicoes on fabric often uses a variety of non-standard inks to make images, but the color separation for this is "obtainable only by very complicated hand works [sic]". In patent parlance, printing of calicoes with special ink sets was admitted as prior art. To get a patent over prior art, one needs to demonstrate a novelty over the prior art which is non-obvious. One of the novelties in this patent is that they laid out an automated mechanism for replacing this very complicated hand works [sic].

Simple, easy to implement color separation technique

I was unsuccessful in finding out whether this invention was ever put to use. Google doesn't seem to know much about it. I have found precious little mention of the patent in my extensive search of technical papers on expanded gamut. I found the patent only through some pretty deep patent searching. The idea may have been actually turned into a product, but it certainly didn't become wildly successful.

Why did this innovation happen? According to the patent: "Such the special color inks [sic] are demanded frequently for the printing of color images which are difficult to produce by the combination of the standard inks."

Why didn't this innovation spread like wildfire? I can only guess, but it might be that the invention lacked a strong market driver. Yeah, it's nice to make more colorful images, but you can't build a better mousetrap until the world beats a path to your door.

Hang on to this thought, though. This patent describes something very similar to what we today call expanded gamut. The color separation is a bit different than what is done today. Currently, colors that are within the CMYK gamut are often printed with just CMYK, but otherwise, this looks a lot like what is done today.

Harald Küppers

Harald Küppers (often spelled either Kueppers or Keuppers, and rarely as kippers) developed a strategy for printing with more than CMYK, and filed for a patent in 1985. His method called for printing "… whereby the elemental surfaces which form the chromatic component are printed with a maximum of two of six chromatic printing inks, yellow, magenta-red, violet-blue, cyan blue, green and black…"

This deserves a bit of explanation. Kueppers' technique was not the traditional halftone technique as we all know and love. In his system, there is no overprinting of the halftone patterns of multiple inks. Instead, Kueppers divided the printed page into square cells, with each cell being divided into rectangular areas of up to four non-overlapping inks. Each rectangular cell is printed with certain rectangular areas of white and black -- the white area may be actually printed, or it may be the color of the substrate showing through. This gives the lightness. Each rectangular cell also is printed with up to two chromatic colors including yellow, magenta-red, violet-blue, cyan blue, and green.

The image below was taken from Kueppers' patent (4,812,899). It demonstrates the conversion of a color value (in this case described as a combination of violet, green, and orange) into a mosaic of printing inks. The colored box in the lower right corner is my own clarification of the box in his patent labelled Fig. 5c. Yeah, the colored box that looks like a Mondrian painting.

Keuppers' printing is a tiling of Mondrian blocks

Note that the little box labelled "S 25" is black (K 25). The German word for black is schwarz. In case you hadn't guessed, Kueppers lived in Germany.

If you squint real hard, this Mondrian of Kueppers tiles will look dark yellowish green

Anyone who has run a printing press will readily recognize that this technique is not practical for litho or flexo printing. Any small amount of misregistration will cause a color shift. For example, if the green ink were to be shifted a bit to the left, it would overlap with black. The color of the overlap would also be black, so misregister in that direction would cause a loss of green.

It is clear that Shimada-san's patent is more closely related to today's expanded gamut printing than Kueppers' patent. And yet, I have seen several references that claim that Kueppers' work is the forerunner of modern expanded gamut printing. Clearly that's wrong, since Shimada-san's invention not only predates Kueppers' work by almost 20 years, but Shimada-san's invention works a lot more like modern expanded gamut printing.

Why has history incorrectly attributed Kueppers with the invention of expanded gamut printing? I have several possible explanations:

1) Stigler’s Law of Eponymy: “No scientific discovery is named after its original discoverer.”

2) Kueppers' idea saw some market success. He produced a color matching book using his process, and apparently a bunch of these were sold. Hallmark's idea was forgotten since it was a niche solution to a different problem. Shimada-san's technique apparently did not create much hubbub. 

3) Kueppers' patent is a bit hard to understand. Much like Bob Dylan's lyrics, I think if Kueppers' expanded gamut printing were better understood, it would have gotten less credit. I readily admit to not understanding the patent the first 413 times I read it. I had been reading it with the assumption that it used normal halftone printing, so the diagram above (Fig. 5) was confusing. I assume that other print historians made the same mistake that I did. 

I didn't catch the idea of Kueppers' patent until I was reading through Kiran Deshpande's doctoral thesis, and saw his Mondrian diagram. Thank you, Kiran!

So, now my two innovation questions for the Kueppers approach to expanded gamut printing.

Why did this innovation happen? According to Kueppers' patent, he sought to solve the problem of moire patterns that are seen with conventional halftone printing, and he wanted solve the problem with conventional printing that "pure and luminous colors cannot be produced well." That is, he wanted to expand the gamut.

Why didn't this innovation spread like wildfire? The references below state that Kueppers' method languished because the color separations were manually intensive. I don't think that this is the case. The patent shows a scanner and a circuit for color separation. It is not necessary to have built an invention in order to get a patent, but it seems that the mechanics were fairly well developed when the patent was filed.

I think the more likely reason that the technique did not flourish is that, as stated before, it just was not practical for typical printing presses because of color shift due register.

Everything you thought you knew about the history of expanded gamut printing is wrong

Clearly, expanded gamut's time had not yet come. I'm getting tired now, so I will continue this in another blog post, which will be called the Heyday of Expanded Gamut Printing Patents


Bernasconi, Mathew, Color Printing Process and Product, US Patent 5,751,326, filed April 5, 1995

Boll, Harald, Color-to-colorant transformation for a seven ink process, Proc. SPIE 2170, Device-Independent Color Imaging, (15 April 1994)

Boll, Harold, and Scott Gregory, Color-to-ink transformation for extra-quarternary printing processes, US Patent 5,563,724, filed Oct 21, 1994

Cooper, Ted, Process for creating five to seven color separations used on a multicolor press, US Patent 5,687,300, filed March 27, 1995

Deshpande, Kiran, N-colour separation methods for accurate reproduction of spot colours, PhD thesis, University of the Arts London, May 2015

Guyler, Karl, Visualization of Expanded Printing Gamuts Using 3-Dimensional Convex Hulls, TAGA 2000

Herbert, Richard and Al DiBernando, Six-color process system, US Patent 5,734,800, filed Nov 29, 1994

Hutcheson, Don, Hi-Fi Color Growing Slowly, GATF 1999

Küppers, Harald, Printing process where each incremental area is divided into a chromatic area and an achromatic area and wherein the achromatic areas are printed in black and white and the chromatic areas are printed in color subsections, US Patent 4,812,899, filed Jan 29, 1989

Küppers, Harald, Process for manufacturing systematic color tables or color charts for seven-color printing, and tables or charts produced by this process, US Patent 4,878,977, filed Nov. 7, 1985

Hamilton, Jim, High Fidelity Seven Ink Printing, Technical document from Linotype-Hell, 1994

Shimada, Shoichi, Apparatus for production of color separation records, US Patent 3,555,262, May 7, 1968

Viggiano, J A Stephen  and William J Hoagland, Colorant Selection for Six-Color Lithographic Printing, Proceedings of the IST/SID 1998 Color Imaging Conference, p 112 - 115

Tuesday, April 3, 2018

A Tale of capitalism with a little twist

Spoiler alert - this story has a surprise ending. That's all I'll say for now.

My family had a Monopoly board when I was growing up. I was the youngest of three, so I got used to ending the game in tears of frustration and humiliation when I landed on Pacific Avenue and had to surrender all my cash and properties over to my greedy sister.

I always lost. It seems I had some very bad character traits. When I landed on Boardwalk, and saw that it would cost $350 of the $600 that I had in front of me, I would opt for frugality. And when my sister pointed out that Illinois Avenue only cost me $240, and she was willing to pay me $300, I opted to be trusting and sell it to her. In the very unlikely event that I had developed a piece of property that my sister landed on, I was often lenient, letting her off with the rent for the undeveloped property. Frugality, trust, and compassion. Poor life choices on my part, indeed.

It looks like I may not be collecting that $200

(I have a sneaking suspicion that my sister may have a somewhat different memory of our board games. Whatever she tells you, just remember that I was the injured party.)

As I grew up, I eventually learned that the best Monopoly strategy was to buy every property that I possibly could, mortgage myself to the hilt to buy properties, and to be ruthless when it came to bargaining with other players. And here's an advanced tip: When all the properties have been purchased, it is better to sit in jail and collect rent rather than use the Get out of Jail Free card. All very important life skills that I learned from Monopoly. I had been indoctrinated into capitalist society.

The History of Monopoly
Everything I know about capitalism I learned from Monopoly

Monopoly was and is an immensely popular game. According to Time Magazine, "Monopoly is the most popular board game in history, with more than 250 million copies sold." If you don't believe Time, the Parker Brothers' website says that Monopoly is "the world's favorite family game brand!" I mean, Parker Brother's should know about their own game!

(Warning: the following few paragraphs contain some information which, for want of a better phrase, are outright lies. I promise to clear it up.)

The story of how this game came to be is a real-life rags-to-riches kinda story. Monopoly was invented by Charles Darrow during the Great Depression. You know... the one caused by ruthless investors who realized that their best strategy was to buy every stock they possibly could? Darrow had lost his job in sales, and pitched the game to both Milton Bradley and Parker Brothers. Initially, they both turned it down.

Ever resourceful, Darrow manufactured the game on his own and sold a very respectable number of copies during the 1934 Christmas season. His sales were enough to bring Parker Brothers back to the bargaining table. Parker Brothers eventually purchased the game and helped him get a patent. Darrow became the first person to become a millionaire by designing a board game. 

There we have it. The game of monopoly working out in real life. A resourceful and tenacious inventor with a good idea gets rich.

Some backstory

Remember how I said there would be a surprise ending? Before you get all warm and fuzzy about the wonderful Charles Darrow, I need to fill in a bit more of the backstory. Then I will get to the surprise ending.

First bubble to burst. Charles Darrow did not invent the game. It seems that Darrow and his wife Esther were invited to the home of Charles and Olive Todd in 1933 to play a board game. Darrow later bugged Todd for details on the game, and for a copy of the board and game cards. The game that Darrow sold to Parker Brothers was a direct copy of the game that Darrow badgered out of Todd. Darrow copied the game right down to a misspelling of Marven Gardens, in the game from ToddMarven Gardens is a subdivision in Margate City, NJ, which abuts Ventnor City, but you will note that in Monopoly, it is spelled Marvin Gardens.

Marvin versus Marven

I don't want to imply that there was anything wrong with Darrow manufacturing a game that was in the public domain, or selling that game to Parker Brothers for a big bunch of money. This is capitalism in action. The fact that the Todds never again invited the Darrows over for Saturday night games was just sour grapes.


Darrow told Parker Brothers that he had invented the game. That was a lie, which is morally icky. It's also a bad business practice to lie to business partners (my opinion), and it opened Darrow up for the possibility of a civil lawsuit from Parker Brothers. I do not consider this to be an example of capitalism at its finest.

Here is Darrow's description of the invention of Monopoly:

"Friends visiting our house in the later part of 1931 mentioned a lecture course they had heard of in which the professor gave his class scrip to invest and rated them on the results of their imaginary investments. I think the college referred to was Princeton University.

Being unemployed at the time, and badly needing anything to occupy my time, I made by hand a very crude game for the sole purpose of amusing myself.

Later friends called and we played this game, unnamed at that time. One of them asked me to make a copy for him which I did charging him for my time four dollars. Friends of his wanted copies and so forth."

Parker Brothers got a bit concerned when one of their VPs recalled a patent from 1904 for a very similar board game. Rather than sue Darrow, they asked Darrow to sign an affidavit to the effect that he was the rightful inventor of Monopoly. He signed it, thereby covering Parker Brothers' butt. 

Note that Darrow also filed for a patent of the game of Monopoly, which is legally a statement of inventorship. With these two legal documents -- the affidavit and the patent application -- Darrow  crossed the line to what I think is criminal activity. Now, I'm not a lawyer, so I can't give legal advice. All I can do is give illegal advice, and write that advice on an illegal pad. But here is my illegal advice: please check with your lawyer before signing any legal documents that you know to be lies. You don't want to be sent to bed without your supper.

Real computer programmers don't like goto statements

Tracing Monopoly backward

The Todds didn't invent Monopoly either, and they never claimed to. In sworn testimony, Charles Todd said that he learned the game from a classmate, Eugene Raiford, who in turn, learned the game from his brother Jesse Raiford.

Jesse was a real estate agent in Atlantic City. Of course, you know that all the properties in Monopoly are named after places in Atlantic City? Jesse Raiford is the guy who was responsible for putting reasonable purchase prices and rents for all the properties, based on his knowledge of the actual streets in question. So, when you buy Indiana Avenue for $220, and have to pay out $1300 when you land on Park Place with four houses, you can thank (or blame) Jesse. 

But he did not select which Atlantic City streets (from Baltic to Boardwalk) were immortalized in Monopoly. A teacher at the Quaker School by the name of Ruth Hoskins had learned of a game on a trip to Indiana. She brought the game back to Atlantic City, where the Quakers adapted the game to the Atlantic City neighborhoods that they were familiar with.

A detailed description of the history of the game prior to Atlantic city can be found elsewhere. I will content myself to jump all the way back to the square labelled "Go".

The Landlord's Game

In March of 1903,  Lizzie Magie filed for patent for what what she called "the Landlord's Game". The drawing below, from the patent, is of the game board. It shows that many of the aspects of the game that we all know and love data back to 1903. The upper left hand square instructs one to "GO TO JAIL". Directly opposite of this, in the lower right corner, is the jail. The square that we sophisticated 21st century beings call "FREE PARKING" was called "PUBLIC PARKING" in the original.
The original Monopoly board

Between each of the corners, we see nine spaces (just like today's Monopoly board), most of which have a sale price and a rent price. The middle space on all four rows is a railroad - exactly the same as the modern board. Utilities (light and water) each have a space, and the original board has not one, not two, but three spaces where one had to pay for luxury

The square that we call "GO" bears an odd label in the original version: "Labor upon Mother Earth Produces Wages". This is the square where the player receives his wages for "perform[ing] so much labor upon mother earth". 

Much of the play proceeds quite similar to the modern version, with the winner being the one with the most money after a predetermined number of trips around the board.

Pausing for some perspective

It is patently obvious that the game that Charles Darrow sold to Parker Brothers was largely derived from the 1903 patent by Lizzie Magie. Clearly Darrow lied about how the game was invented. There can be no question about that. But there are two other pertinent questions to address before I continue.

Did Parker Brothers violate Magie's patent?

No. At that time, patents lasted for 17 years after they were granted. (Today, the term is 20 years after the patent is filed.) This patent was granted in 1904, so it expired in 1921. After that time, the invention in the claims entered into the public domain, so anyone was free to make or sell Magie's board game in 1934.

Does the Magie patent invalidate Darrow's patent? 

It would be natural to think that, once "Monopoly" has been patented, the game is over, and it can't be patented again. But one of the cool things about patents is that the patent office is cool with you filing for a patent on an improvement to an existing patent. They even encourage it. Your improvement just can't be obvious when compared with any prior art.

The Magie patent does not describe naming of the properties, grouping properties by color, or rents that depend on additional investments into a property, or the need to own all of the same-colored properties before developing the property. So, if the patent examiner found this to be non-trivial, then there was room for additional patents. I readily admit that I have not spent the week or so necessary to fully understand the claims in Darrow's patent.

Magie's second patent

Elizabeth Magie Phillips (AKA Lizzie Magie) filed for a second patent on her Landlord's Game in 1923. There are some changes to the game. While the new properties have been given fictitious street names, the layout of the board departs from the original patent. The current Monopoly board is clearly derived from Magie's first patent, not from her second.

There are some additions which have made this filing patentable over the previous patent. The new set of claims includes the notion of a grouping of properties together. I would presume that this is a novelty which distinguishes the two patents. But again, I fully admit to not investing a lot of time into interpreting one set of claims over the previous disclosure. 

Magie's second patent was granted in 1924, which means that it was in effect when Parker Brothers and Darrow were signing their deal. Uh-oh. But, Parker Brothers did their due diligence, and negotiated with Magie over the rights for her second patent. She received $500.

That number might just tick you off, but hear me out...

The Monopoly game we all know and love is based on Magie's original patent, so you may feel that she has a right to any and all profit from the Monopoly game. But, she had her chance, between 1905 and 1921 to have a monopoly on Monopoly.

Another consideration is that $500 is not all that much money today. Adjusted for inflation, that $500 would be worth $9,300 today. Not a bad chunk of money. Considering the cost and effort of developing a successful product from a good idea, maybe the size of the check was reasonable?

A final consideration is that there is a real question whether the existing Monopoly game would infringe on the 1924 patent. As with all patents, it comes down to how to interpret all the elements of a claim. For example, claim 1 of the 1924 patent includes "a series of cards of changeable value, two or more of which are alike and which relate to two or more certain spaces on the board". If Monopoly can be shown to not have such changeable cards, then it does not infringe claim 1. To be honest, I don't know what the phrase means. Given enough time, I'm sure I could work up an argument either for Parker Brothers or for Magie.

So, Magie may have been able to parlay this patent into a lot more money, but probably not without  fair amount of legal expense and a certain risk. Collecting $9,300 may well have been a reasonable choice.

And that's capitalism for you.

One more thing...

One of the topics of this blog post is capitalism. I would be remiss if I didn't share a bit more about Elizabeth Magie and some wording from her second patent.

Magie describes the purpose of the game as follows: "The object of the game is not only to afford amusement to the players, but to illustrate to them how under the present or prevailing system of land tenure, the landlord has an advantage over other enterprises and also how the single tax would discourage land speculation".

Here are some other political gems in the patent.

1. There is a space on the board named "Lord Blueblood's estate". "This space represents foreign ownership of American soil, and carries with it a jail penalty for trespassing."

2. Another space is call "La Swelle Hotel". "This space represents the distinction made between classes, only moneyed guests being accepted." 

3. In one particular misfortunate toss of the dice, the player will have been caught robbing the public. They will take $200 from the bank, and the other players will thereafter call the player "Senator". To the best of my knowledge, this is the harshest insult to be found in all the patent archives.

Clear political undertones. And overtones too, for that matter!

Magie's intent with the game was to illustrate the inequity inherent in the idea of people getting paid by rent, beyond pay for actual labor. In a sense, you could say that she was using the game to indoctrinate the players into this anti-capitalistic idea. You see, Elizabeth Magie was a Georgist, one who believes that people should earn money from hard work, but not from property or natrual resources.

The irony is that the game Monopoly has been used for several generations as a training mechanism for young capitalists. That's my surprise ending.


Cheating at Monopoly: Uncovering the secret history of the classic board game

Stealing Monopoly

The Culture Complex: Monopoly Is Us

MONOPOLY: From Berks to Boardwalk

The fake history — and the real one — behind the inventing of ‘Monopoly’

Sunday, April 1, 2018

Scientists discover new astrological sign

NASA scientist Ben Capricorn announced today the discovery of a thirteenth sign of the zodiac, which has been tentatively named Naivius the Confounder. The constellation for this sign differs from all others in that it spans the entire 360 degree sky.

Naivius the Confounder is obvious, once you see it

Dr. Capricorn explained that he was studying anomalies in horoscopes, people who did not match their signs. "I had been pondering a blog post by John the Math Guy which showed that the signs of the zodiac are useless in predicting mathematical genius. It suddenly occurred to me that there must be another celestial influence which has some effects that were not seen by Ptolemy, who codified these laws a millennium or so ago." Dr. Capricorn theorized that the influence must have been from heavenly bodies that were not known at the time of the discovery of astrology.

Dr. Ben Capricorn of NASA's Office of Space

So, Capricorn petitioned NASA for time on the Hubble telescope to peer deeply into the twelve constellations to find occult stars that might explain the anomalies. Simon Rasputin, director of the Government Office of Pseudoscience applauds this effort. "Capricorn's work is far beyond anything that cosmologists have been able to piece together with all their silly-talk about black holes and the red shift and the big bang and all that stuff. It make so much more sense to group stars that are billions of light years apart into constellations [rather than group them according to the groups of stars that are near together and which rotate about a common center of mass.]  Mystical forces are way more better than dumb equations."

Capricorn's theory was proven true. He was able to find minor stars which correspond to each of the anomalous mathematicians who were not Virgos, as all real math guys should be. These minor stars together form the constellation Naivus the Confounder.

This confirmation of Dr. Capricorn's theory is just the start of this momentous task. "I have already applied for a grant to continue this work. For the project, I will investigate the horoscopes of each and every one of the 7.4 billion people on the Earth, and find an occult star among the hundreds of billions of stars to explain why 91.7% of all people don't fit their horoscope. The project is staggering in it's proportions, but the ultimate benefit to humankind is immeasurable."

Wednesday, February 28, 2018

Is my color process all wonky?

In a previous post, I looked at how the Zc statistic can be used to isolate individual color measurements that are icky-poopy. Today I look at a slightly broader question: How can I tell if the whole production run is wonky?

I think something went wrong in this production run

I deliberately use the word "wonky" because it has a squishy meaning, which is helpful, since I'm not sure what I want it to mean just yet! So, bear with me while I fumble around in the darkness, not knowing quite what I am doing just yet. 

Here is the premise: Color, in a well-controlled production run, should vary according to some specific type of statistical distribution. (Math mumbo-jumbo alert) I will take a guess that the cloud of points in that ellipsoid of L*a*b* values is a three-dimensional Gaussian distribution, with the axes appropriately tilted and elongated. If this is the case, then the distribution of Zc will be chi with three degrees of freedom. (End math mumbo-jumbo alert.)

If you are subscribed to the blog post reader that automatically removes sections of math mumbo-jumbo, then I will recap the last paragraph in a non-mumbo-jumbo way. In stats, we make the cautious assumption of the normal distribution. Since I am inventing this three-dimensional stats thing, I will cautiously assume the three-dimensional equivalent. But, since this is virgin territory, I will start by testing this assumption.

A quick note about CIELAB target values and DE

This blog post is not about CIELAB target values and DE. Today, I'm not talking about assessing conformance, so DE is not part of the discussion. I am talking about whether the process is stable, not whether it's correct.

A look at some real good data

Kodak produced a photographic test target, known as the Q60 target, which was used to calibrate scanners. The test target would be read by a scanner, and the RGB values which were read were compared against L*a*b* values for that batch of targets in order to calibrate the scanners. When the scanner encountered that same type of film, this calibration would be used to convert from RGB values to moderately accurate L*a*b* values. Hundreds of thousands of these test targets were manufactured between 1993 and 2008.

I think the lady peeking out on the right is sweet on me

We know that these test targets were produced under stringent process control. They were expensive, and expensive always means better. More importantly, they were produced under the direction of Dave McDowell. I have worked with him for many years in standards groups, and I know they don't come more persnickety about getting the details right than him!

Dave provided me with data on 76 production runs of Ektachrome, which was averages of  the L*a*b* values from 264 patches, for a total of about 20K data points. So, I had a big pile of data, collected of production runs that were about as well regulated as you can get.

I applied my magic slide rule to each set of the 264 sets of 76 color values. Note that I pooled at the data for individual colors of patches. General rule in stats: You don't wanna be putting stuff in the same bucket that belongs in different buckets. They will have different distributions.

Within each of the 264 buckets, I computed Zc values. Twenty thousand of them. I hope you're appreciative of all the work that I did for this blog post. Well... all the work that Mathematica did.

Now, I could have looked at them all individually, but the goal here is to test my 3D normal assumption. I'm gonna use a trick that I learned from Dave McDowell, which is called the CPDF.

Note on the terminology: CPDF stands for cumulative probability density function). At least that's the name that it was given in the stats class that I flunked out of in college. It is also called CPD (cumulative probability distribution), CDF (cumulative distribution function), and in some circles it's affectionately known as Clyde. In the graphic arts standards committee clique, it has gone by the name of CRF (cumulative relative frequency).

Here is the CPDF of the Ektrachrome data set. I through all the Zc values into one bucket. In this case I can do this. They belong in the same bucket, since they are all dimensionless numbers... normalized to the same criteria. The solid blue line is the actual data. If you look real close, you can see a dotted line. That dotted line is the theoretical distribution for Zc that 3D normal would imply. Not just one particular distribution -- the only one.

20,000 color measurements gave their lives for this plot

Rarely do I see real world data that comes this close to fitting a theoretical model. It is clear that L*a*b* data can be 3D normal.

More real world data

I have been collecting data. Lots of it. I currently have large color data sets form seven sources, encompassing 1,245 same-color data sets, and totalling 325K data points. When I can't sleep at night, I get up and play with my data. 

[Contact me if you have some data that you would like to share. I promise to keep it anonymous. If you have a serious question that you want to interrogate your data with, all the better, Contact me. We can work something out.] 

I now present some data from Company B, which is one of my anonymous sources. I know you're thinking this, but no. This is not where the boogie-woogie bugle boy came from. This complete data set includes 14 different printed patches, sampled from production runs over a full year. Each set has about 3,700 data points. 

I first look at the data from the 50% magenta patch, since it is the most well-behaved. The images below are scatterplots of the L*a*b* data projected onto the a*b* plane, the a*L* plane, and the b*L* plane. The dashed ellipses are the 3.75 Zc ellipses. One might expect one out of 354 data points to be outside of those ellipses.

Three views of the M 50 data from Company B

Just in case you wanted to see a runtime chart, I provide one below. The red line is the 3.75 Zc cutoff. There were 24 data points where Zc > 3.75. This compares to the expectation of 10.5. This is the expectation under the assumption that the distribution is perfectly 3D normal. I am not concerned about this difference; it is my expectation that real life data will normally exceed the normal expectations by a little bit.

Another view of the M 50 data - Zc runtime plot

So far, everything looks decent. No big warning flags. Let's have a look at the CPDF. PArdon my French, but this looks pretty gosh-darn spiffy. The match to the theoretical curve (the dotted line) is not quite as good as the Ektachrome data, but it's still a real good approximation. 

Another great match 

Conclusion so far, the variation in color data really can be 3D normal!

Still more real world data

I show below the CPDF of Zc for another data set from that same source, Company B. This particular data set is a solid cyan patch. The difference between the real data and the theoretical distribution is kinda bad.

A poor showing for the solid cyan patch

So, either there is something funky about this data set, or my assumption is wrong. Maybe 3D normal isn't necessarily normal? Let's zoom in a bit on this data set. First, we look at the runtime chart. (Note that this chart is scaled a bit different than the previous. This one tops out at Zc = 8, whereas the other goes up to 5.5.) 

A runtime chart with some aberrant behavior
that will not go unpunished!

There are clearly some problems with this data. I have highlighted (red ellipse) two short periods where the color was just plain wonky. Some of the other outliers are a bit clustered as well. Below I have an a*b* scatter plot of that data (on the left), and a zoomed-in portion of that plot which shows some undeniable wonk. 

Look at all the pretty dots that aren't in the corral where they belong
I'm gonna say that the reason that the variation in this data set does not fit the 3D normal model is because this particular process is not in control. The case gets stronger that color variation is 3D normal when the process is under control.

Are you tired of looking at data yet?

We have looked at data from Company K and Company B. How about two data sets from Company R? These two data sets are also printed colors, but they are not the standard process colors. There are  about 1,000 measurements of a pink spot color, and 600 measurements of a brown spot color. One new thing in this set... these are measurements from an inline system, so they are all from the same print run.

First we look at the CPDF for the pink data. Yes! I won't show the scatterplots in L*a*b*, but trust me. They look good. Another case of "3D normal" and "color process in good control" going hand-in-hand.

Yet another boring plot that corroborates my assumptions

Next we see the CPDF of Zc for the brown data. It's not as good as the pink data, or the Kodak, or the M 50 CPDF plots, but not quite as bad as the C 100. So, we might think that the process for brown is in moderate control?

Brown might not be so much in control?

The runtime chart of Zc looks pretty much like all the others (I could plop the image in here, but it wouldn't tell us much). The scatter plots of L*a*b* values also look reasonable... well, kinda. Let's have a look.

Halley's comet? Or a scatterplot of variation in brown?

.This data doesn't look fully symmetric. It looks like it's a little skewed toward the lower left. And that is why the CPDF plot of brown looks a bit funky. Once again, we see that the CPDF of Zc values for a set of color variation is a decent way to quickly assess whether there is something wrong with the process.

But why is the brown plot skewed? I know the answer, but we're gonna have to wait for the full exposition in another blog post.

For the tine being, let me state the thrilling conclusion of this blog post.

The thrilling conclusion of this blog post

When a color producing process is "in control" (whatever that means), the variation in L*a*b* will be 3D normal. This means that we can look at the CPDF of Zc as a quick way to tell if we have exited the ramp to Wonkyville.

Monday, February 19, 2018

Munsell - the Father of Color Science? (part 3)

This series of blogs was foretold in a prophecy of April of 2013:

Someday I will write a blog post about how this guy Munsell laid the foundation for the ever-popular color space CIELAB, and came to be known as the Father of Color Science. He was also the father of A. E. O. Munsell, who carried on his work. I don't intend to write a blog post about how Albert became the father of A. E. O.

What I did not foretell in that blog post is that ISCC will be sponsoring the Munsell Centennial Symposium,  June 10 - 15, 2018 in Boston. Or that I would be keynoting this event.

After two previous attempts (Munsell as an educator, Munsell and 3D color space), I am finally have made my way to looking at the most significant work of Munsell. The Munsell Color Space was a model for CIELAB.

First cursory pass

Exhibit A. Richard Hunter's book The Measurement of Appearance, on page 136.

Photo taken at the Color Difference family picnic

This is a family tree of proposed models for determining color difference. Note that the Munsell Color system is in the upper right hand corner, and all arrows come from that box. The only little boxes that are still active today are the two boxes labeled CIE 1976. A similar diagram is on page 107 of that same book, which shows a family tree of color scales. (I have an image of that in a previous blog posts about color difference.) Again, this shows a straight lineage from Munsell Color Scpce to CIELAB.

Is this reliable testimony? Richard Hunter was a fairly knowledgeable guy when it comes to color. I mean, he has his own entry in Wikipedia for goodness sake. CIELAB is (perhaps) the most widely used tool in the color industry. Since Hunter traces the lineage of CIELAB back to Munsell, then I feel pretty confident about putting Munsell on the shortlist of highly influential figures in the history of color science, at the very least.

But, that hides a lot of the fun stuff that happened between the creation of the Munsell color space and the ratification of CIELAB as a standard for color measurement.

What Munsell did

    Munsell Color Space

Munsell's color space is based on some simple principles.

1. Hue, Value, and Chroma

There are three attributes to color in Munsell's color system. While these are implicit in many of the previous color systems (enumerated in a previous blog post), Munsell was intent on tying these to our intuitive understanding of color. (After studying on this for 25 years, I have come to realize that they are indeed intuitive). 

2. A physical standard produced with simple tools, simple math, and a defined procedure

Munsell described the procedure by which his color system could be developed from any reasonable set of pigments. The procedure included a way to assign unique identifiers to each color.  As a result, all colors within the gamut of the chosen pigments could be unambiguously named.    

3. Perceptual linearity

One of Munsell's secondary aims was to create a color space where the steps in hue, value, and chroma were all perceptually linear. Did he meet his goal? Stay tuned.

This color system was used to create the Atlas of the Munsell Color System, which was a book containing painted samples with their corresponding designations of hue, value, and chroma. This book was to be used as an unambiguous way to identify colors, and thus, to provide a standrd way to communicate color.

     Munsell photometer and the gray scale

Munsell invented and patented a photometer which was capable of measuring the reflectance of a flat surface. Well, provided it was a neutral gray. The user would look into a box and see two things: the sample to be measured, and a standard white patch. The sample was illuminated with a constant illumination, and the white standard was illuminated with light through an adjustable aperture. To make a measurements, the size of the aperture was adjusted so as to match the intensity of the dimmed white standard and that of the sample. The width of the aperture, scaled from 1 to 10, was the Munsell Value for the gray sample.

A shoebox with some holes and stuff

Munsell used his photometer to mix black and white paints in steps from V = 1 to V = 10.

     Maxwell disks and the rest of the colors

James Clerk Maxwell invented a creature called the Maxwell disk around 1855. I spent the better part of a day building my own set of Maxwell disks from colored construction paper as shown below. The cool part is the slit. You can slide two or more disks together, and rotate them so as to get any proportion of the colors to show. In the inset, I show the device that I adapted to rotate the disks. Again, the better part of a day was spent assembling a bolt, a couple of washers, and a nut. I first tried a cordless drill, and found it didn't spin fast enough to merge the colors. I had to use my old drill that plugs into the wall.

The Maxwell disks were the inspiration for PacMan

The picture below shows the results of day 3 of my dramatic reenactment of Munsell's landmark experiment. I selected red, green, and blue construction paper, and adjusted the size of the segments in order to get a facsimile of gray. When I saw that gray, I realized that this was four days well spent.

Me, geeking out on the creation of gray from Red, green, and blue

If I were to be doing this on a government grant, I would have spent another day or two actually measuring the sizes of the red, green, and blue areas. For the purposes of this blog, I will be content with just saying that red and blue are each one-quarter, and green is one-half. In other words, this green is half as strong as the others. Thus, Munsell would conclude that the chromas of this red and blue were twice that of this green. Munsell would also have measured this gray with his photometer. Another opportunity for me to get a little more grant money.

In this way, Munsell was able to assign values to the colors.

     Perceptual linearity?

Linear in Value?

Since Munsell's original Value was measured as the width of an aperture, the amount of light let through is proportional to the square of the Value. Conversely, Value is proportional to the square root of the light intensity. The plot below compares this scale against today's best guess at perceptual linearity, CIEDE2000.

Munsell's original Value was kinda sorta close to perceptually linear

Note: The DE2000 scale in the plot above is based on Seymour's formula (L00 = 24.7 Log e (20 Y +1), where 0< Y < 1), which was first presented at TAGA 2015, Working Toward A Color Space Built On CIEDE2000. The height of the curve at the end shows that there are 76 shades of gray, based on DE2000. The Munsell Value has been scaled to that.

Is this perceptually linear? That depends on how gracious you want to be. On the one hand, the linearity is not lousy. Given the tools at hand, Munsell did a fairly decent job of making kinda linear.

On the ungracious side, Munsell merely took what he had handy (the size of the opening of his aperture) and used that. Lazy bum! Surely he would have known about the work of Ernst Weber (1834) and Gustav Fechner (1860) which postulated that all our perception is logartihmically based! 

Really pedantic note: There is some confusion about how the gray scale was set up. My description is based on Munsell's description [1905], as well as comments by Tyler and Hardy [1940], Bond and Nickerson [1940], and Gibson and Nickerson [1940], all of which were based on Munsell's words and measured samples. But in a paper from 1912, Munsell described his assignment of Value as being logarithmic, following the Weber-Fechner law.

Linear in hue?

Munsell started this exercise by selecting five paints with vibrant colors: red (Venetian red), Yellow (raw sienna), green (emerald green), blue (cobalt), and purple (madder and cobalt). He then created paints that were opposite hues for each of these. The opposite hues were adjusted so that the balanced out to gray on the Maxwell disks. Thus, he had a set of ten colors with Value of 5 and Chroma of 5.

What's to say that these paints are equally spaced in hue? I am sure that Munsell selected them with that in the back of his mind, but four of the five are just commonly available, single pigment paints.
From the literature that I reviewed in the bibliography below, I could find no evidence that he put much time into psychophysical testing.

I'm gonna say that the hue spacing in the original Munsell color system is only somewhat perceptually linear.

Linear in Chroma?

Munsell's assignment of Chroma values is all based on simple ratios of areas on the Maxwell disks. Thus, in his original system, chroma is linear with reflectance. I did a bit of testing, comparing Munsell's proposition against DE2000. I will smugly state that our perception is not linear with reflectance.

But Munsell begs to differ with me. He performed some tests of this, and summarized his results in 1909:

These experiments show clearly that chroma sensation and chroma intensity (physical saturation) vary not according to the law of Weber and Fechner, but nearly or quite proportionately, and in accordance with the system employed in my color notation.

This paper seems to have been largely ignored by other color researchers. Deane Judd looked at the question of equal steps in chroma in 1932. His bibliography included Munsell's 1909 paper, but he made no mention of it in the text. The same with several of the papers from 1940 listed below.
My brief test suggests this is not true, and the people who were genuinely interested in the question who were aware of Munsell's suggestion ignored it. The graphs from the 1943 paper (Newhall, et al.) are decidedly non-linear in steps of chroma. Barring further evidence, I would say that the original Munsell Color System was not perceptually linear in chroma.

All in all, I'm gonna rate the claim that the original Munsell system was perceptually linear as "Mostly False".

What happened after Albert Munsell

Albert Munsell passed on in 1918, but a lot of work was done on the Munsell Color System by others after his death.

In 1919 and again in 1926, Munsell's son, A. E. O. Munsell submitted samples to the National Bureau of Standards. These were measured spectrophotometrically. The 1919 data was analyzed by Priest et al., and came along with some suggestions for improvement. They suggested that the Value scale be changed. 

This challenge was taken up by Albert's his own son. In 1933, A. E. O. published a paper describing a modification of the function from which Value was computed. This brought value much closer into line with the predictions of CIEDE2000.

The Munsell Color System was largely ignored in the literature until 1940. At that time, seemingly everyone jumped on the bandwagon. A subcommittee of the Optical Society of America was formed, and the December 1940 issue of the Journal of the Optical Society in America published five papers on the Munsell Color System.

Why the sudden effort? Spectrophotometers were expensive and cumbersome, but were becoming available. The 1931 tristimulus curves were available to turn spectral data into human units. Several of the papers noted a desire to create a system which translated physical measurements into something that made intuitive sense.

The Munsell Color System seemed to be best template to shoot for, since it was "[l]ong recognized as the outstanding practical device for color specification by pigmented surface standards."  (Newhall, 1940)

The efforts of the OSA subcommittee culminated in what has become known as the Munsell Renotation Data, introduced in the 1943 paper by Newhall et al. Inconsistencies of the original data were smoothed out, a new Value scaled was introduced, and a huge experiment (3 million observations) was done to nudge the colors into a system that looked perceptually linear.  The final result is a color system that can indeed be said to be perceptually linear.

Oh what a tangled web we weave, from Newhall (1943)

I'm not gonna take up the rest of the story, from the Renotation Data to CIELAB. That's another long and interesting story, I'm sure. But I am running out of gas!


Here is the firmest entirely factual statement that I can make about this paternity suit involving Albert Munsell and the child named Color Science.

Munsell had a passion for teaching color, especially to children. He sought to bring order and remove ambiguity from communication of color. This passion brought him to create the Munsell Color System. This was not the first three-dimensional arrangement of color, nor was it all that close to being perceptually linear. But it had two great features going for it: It was built on the intuitive concepts of hue, chroma, and lightness, and it came with a recipe for building a physical rendition of the color space. As a result, the Munsell Color Space is both a concept for understanding color, and a physical standard to be used in practical communication of color.

The Munsell Color System saw a number of improvements after his death, resulting in the Munsell Renotation Data. This later became the framework for future development of a magic formula to go from measured specrta to three numbers that define a color. The CIELAB formula is the one that stuck.

I realize that my work over the past 25 years has given me a bias toward the importance of measurement of surface colors, and hence a bias toward thinking that CIELAB is important. The next statement is subjective, and based on my admitted biases.

I think that Albert Munsell deserves to be called The Father of Color Science.

Albert Munsell proudly showing off his very attractive John the Math Guy Award

On the other hand...

I would be remiss if I failed to mention a few other individuals, who might reasonably be on the podium with Munsell.

Isaac Newton - He invented the rainbow, right? Well, actually, he did some experiments with light and came up with the theory of the spectrum. Spectrophotmeters are designed to measure this.

Thomas Young - He first proposed the theory that the eye has three different sensors (red, green, and blue) in 1802. Hermann von Helmholtz built on this in 1894.

Ewald Hering - He proposed the color opponent theory in 1878. Light cannot be both red and green; nor can it be both blue and yellow. His three photoreceptors were white versus black, red vs green, and yellow vs blue. This is explicitly built into CIELAB.

It turns out that all of these are correct, but they are looking at different stages in our perception. Newton's spectrum is a real physical thing. The retina does have three Young-Helmholtz sensors. The cones are not exactly RGB, but kinda. And the neural stuff after the cones in the retina creates signals that follow Hering's theory.

So, maybe one of these gents should get the crown? I dunno... maybe I'll make a few more John the Math Guy awards?


Munsell's papers

Munsell, Albert H., A Color Notation, Munsell Color Company, 5th Edition, 1905, Chap V

Munsell, Albert H., On the Relation of the Intensity of Chromatic Stimulus (Physical Saturation) to Chromatic Sensation, Psychological Bulletin, 6(7), 238-239 (1909)

Munsell, Albert H, A Pigment Color System and Notation, Amer. Journal of Psych, Vol 23, no. 2, (April 1912)

Post Munsell, pre-1940

Priest, Irwin, K. S. Gibson, and H. J. McNicholas, An examination of the Munsell color system. I. Spectral and total reflection and the Munsell scale of value, Tech. Papers of the Bureau of Standards, No. 167 (September 1920)

Judd, Deane, Chromatic Sensibility to Stimulus Differences, JOSA 22 (February 1932)

Munsell, A. E. O., L. L. Sloan, and I. H. Godlove, Neutral Sclaes. I. Munsell Neutral Value Scale, JOSA (November, 1933)

Glenn, J. J. and J. T. Killian, Trichromatic analysis of the Munsell Book of Color, MIT Thesis (1935), also in JOSA (December 1940)

The 1940's flurry

Gibson, Kasson S and Dorothy Nickerson, An Analysis of the Munsell Color System Based on Measurements Made in 1919 and 1926, JOSA, December 1940

Newhall, Sidney, Preliminary Report on the O.S.A. Subcommittee on the Spacing of the Munsell Colors, JOSA, December 1940

Tyler, John E. and Arthur C. Hardy, An Analysis of the Original Munsell Color System, JOSA December 1940

Nickerson, Dorothy, History of the Munsell Color System, Company, and Foundation. II. Its Scientific Application, JOSA, December 1940

Bond, Milton E., and Nickerson, Dorothy, Color-Order Systems, Munsell and Ostwald, JOSA, 1942

Newhall, Sidney M., Dorothy Nickerson, and Deane B. Judd, Final Report of the O.S.A. Subcommittee on the Spacing of the Munsell Colors, JOSA July 1943

More recent

Hunter, Richard S., The Measurement of Appearance, John Wiley, 1975, pps. 106 - 119