Yes but it's not easy to find random Wikipedia user pages, or even find the specific talk page comment in the archives without knowing what to look for. Go ahead, find a friend, give them no clues, and see if they find it.
I don't know what “investing over” means, but all I can say is (repeating myself): try it with a friend, without giving them the benefit of hindsight.
(Very few visitors to a Wikipedia page read its talk page, very few of them will further look at the archives of the talk page, let alone read every single comment and its corresponding commenter's name, and in this case as soon as the author knew the spelling to look for, the rest was straightforward for them.)
well said svat. without laboring the point, "once you know - you know". until then it is like trying to find a needle in a haystack - and some people do not want to be found.
I mean that the author was literally writing a book about this and literally conducting an investigation about who was the real author of the TIFF spec. He interviewed other people about it, he had a name that had a typo in it, he even found a white-on-white line in a PDF with the real name. He was clearly putting effort into the research.
We are humans, everybody can miss things, I mentioned "hindsight is 20/20" but still, it was in the Wikipedia discussion page for the TIFF article all the time. It's a matter of fact and some random HNer found it in minutes/hours.
I repeat myself, it was probably better that he didn't found that out and went to write a hand-written letter to the alleged author's home address, it created a much deeper human bond, which is especially meaningful since Stephen Carlsen passed away not much later.
"The answer to this is very simple. It was a joke. It had to be a number, an ordinary, smallish number, and I chose that one. Binary representations, base thirteen, Tibetan monks are all complete nonsense. I sat at my desk, stared into the garden and thought '42 will do' I typed it out. End of story." from the man himself[1]
...but let us not ruin a good story with the truth. Remember why earth was built. The "real" answer might then be flowing in the ether.
Talk pages aren’t valid sources in general. In this case the author is dead and an established expert having published in the field, so I guess it’s fine, but I wouldn’t bet.
If you had told me an article ostensibly about a file format would have me teary-eyed by the end I wouldn't have believed you. This is beautiful, thank you!
A huge number of us labor behind the scenes with no public acknowledgment or credit. For each idea brought to life we might hope for -at most- an epitaph carved in expired patent claims.
thanks daniel. i swapped a few emails with stephen's ex-wife and they melted my heart. i asked if she was OK with sharing this story of a humble genius with the wider world and she said, yes.
It is, indeed. Thanks for writing that up, it touched me.
And thanks for being the historian of our culture-that-eschews culture (or so it seems to me sometimes, that tech tries to exist in a perennial present without acknowledging it's roots and history)
thanks isoprophlex for the acknowledgment. so many of the folks that i have interviewed, contributed so much to making tech or a product better for all. especially at or around apple. they receieved none of the credit that the 'names' got. ;-) you know who i mean.
Thanks! If you look at his (logged-in) edits on Wikipedia (https://en.wikipedia.org/wiki/Special:Contributions/Scarlsen ), then apart from the lone comment on the talk page (about the reason for "42") and creating that user page, he has two edits to the TIFF article:
- and the other is even more interesting: though he is being scrupulous and removing a sentence that has no published citations, in his edit summary he confirms that it is basically true:
> The author of the original TIFF specification wanted TIFF to stand for "The Image File Format", but he was overruled by Aldus' president Paul Brainerd on the grounds that it sounded presumptuous.
(The edit summary says: Removed the "The Image File Format" sentence, since it only has eye-witness support (me, for one), but no published citatations)
> Primary sources that have been reputably published may be used in Wikipedia, but only with care
and I imagine a Wikipedia edit summary does not count as a reliable source. (For one thing, despite it being very plausible that the Wikipedia user Scarlsen who signed himself as Stephen E Carlsen is indeed that person—I believe it completely!—it cannot be guaranteed that it wasn't an impostor, for example.)
That would be what Wikipedia calls "original research". A big no-no on wikipedia. At a minimum he would have to tweet or blog about it and link the tweet or blog. And even then that's a primary source, which wikipedia considers less valuable. Ideally he would get someone else to report on his tweet/blog and use that as source. Then the wikipedia gods are happy
Technically yes, but I'm fairly sure Wikipedia wants cited sources, not "I'm the guy, I said so" anecdotal sources.
Of course, if he was still alive he could have written a blog post or something like that and use that as a source, much like how it's likely this blog post will be used as a source for things surrounding the format and person.
>Yes it is true: the second word of a TIFF file, 42, was indeed taken from the answer to the Ultimate Question of Life, the Universe, and Everything, from Hitchhikers_guide_to_the_galaxy. StephenECarlsen 23:38, 12 October 2007 (UTC)
Hey John, I'm just curious how people find these comments about "would be nice if X saw this" on HN. I don't think there's any pinging behavior. Did somebody message you? Did you just happen to read it? Do you have an eldritch curse that summons you when called by name?
Not who you're asking for, but generally I think it's just a case of the author also being an HN regular. Although, I suppose you could set up some Google Alerts for mentions of your blog posts.
I bet Musk hacked something together (or has a column in TweetDeck if that's still around) that continuously searches for mentions. I wonder if there's a tool like that that covers more of the internet, although the primary users would only be famous people and/or their agents / social media staff.
There are lots of tools that do this if you are willing to pay for it. "Social Media Monitoring" or "Brand Monitoring" would be the keywords to search for
From what I've seen, sometimes people see that their blog post got a lot of referral traffic from Hacker News and they come have a look to see where it came from.
The great thing about TIF was it's extensibility. Flexible (data could be stored as tiles or in stripes), multiple compression options etc.
Well documented spec, easy to bolt on extras either as public tags - GeoTIFFs added projection metadata - or private, for your own needs.
Back in the day, to improve a desktop application's performance I found it was simple to create a custom reader and writer to handle cases where tiles were completely one single colour removing the need to decompress at run time.
Perhaps the greatest thing about TIFF, but also the most horrible things, and probably why TIFF is mostly historical. It was so extendable that no two programs ever accepted the exact same TIFF extensions. (omitting the war story)
But most of these variations were part of the spec (endianness with II or MM, later magic 43 for bigTIFF 64bit extension). I work with tiff and tiff-derived formats in digital microscopy where its very much not historical. And the alternatives (DICOM supp 145, vendor-specific garbage ... and thats it) are worse.
I quite like the format, the only thing I would change is to have the option not to store directory information in a linked list spread throughout the file but in a simple array. Duplicate it at the beginning and end of the file and you've got resilience too (important in the age of floppies)
We have hundreds of thousands of TIFF files where I work which are scans of questionnaires filled out by clinical trial participants. The one annoying thing is that web browsers don’t natively display them. I did some incredibly inefficient JavaScript bs to decode the pixel data, plop it in a canvas, get a PNG data url from it, and set that as the src for an img element xD (why not just display the canvas? because I was too lazy to manually handle resizes…) good times
TIFF is still very much alive in certain circles, see for example https://cogeo.org/
The format is basically a TIFF file with attached georeferencing information and with the data organized by geographical sector, enabling fast downloads of regional subsets.
The Adobe DNG standard for raw camera images is based on TIFF as well. DNG is used in lots of places, including the raw capture support built into all modern iOS and Android smartphones.
I’ve been using both TIFF and DNG this very week in my work (https://filmlabapp.com), so I was happy to read this post and learn about Steve Carlsen aka Mr. TIFF, whose work we’re still building on 39 years later.
Not the .IA files from old Sinar digital backs! Those are based on DOOM PWADs (lol). But otherwise mostly yes this is true for nearly every other format as far as I am aware.
I'm not super knowledgeable about this stuff.. but out of curiosity, what advantage does it have over NetCDF?
I wrote a program processing GeoTIFF data. When I had started this project I chose GeoTIFF mostly b/c i wanted something simple. And I could load them in to Java's BufferImage class and manipulate them that way. But it seems all the pros exclusively use NetCDF and GeoTiffs are for noobs (working with atmospheric science data here)
GeoTIFF does extend "images" to cover more usecases, but a lot of stuff doesn't fit (like say a wind vector) and then you need some other container or metadata b/c you generally have many images. So I get the sense the complexity just ends up being moved elsewhere.
Generally speaking I would classify TIFF and its variants as imaging formats (or for very simple numerical datasets), and NetCDF as more suited for raw data, in particular multi-dimensional data with time series, etc.
For forecast and climatological data I find NetCDF is vastly superior, but also much more complicated to work with due to the capabilities and how open the format is. Just have a look at the complexity of the CF Conventions to see what I mean: https://cfconventions.org/cf-conventions/release/v1.12.0/cf-...
For visualizing orthophotos and the like, I would choose GeoTIFF any day of the week, as they're easy to visualize across platforms using existing libraries. Using COGs you also get the functionality of a spatial index within each GeoTIFF file, meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request.
Yeah the complexity of NetCDF was the ick factor that made me use GeoTIFF. It's just not the level of complexity I wanted to deal with in-program. I didn't want to wade through the details to understand the format, and I didn't want to then lock users to some subset
Seemed easier to let users preprocess their NetCDFs into GeoTIFFs manually. I have a bunch of hacky scripts to massage NetCDFs from different sources in to compatible GeoTIFFs
> meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request.
Interesting. My performance bottleneck right now is the user selects a small regions and then the program has to read in GBs of global precipitation maps (from IMERG) and cut out tiny squares. In the extreme cases it can mean ~2 minutes of waiting for a result. This means the user can't casually select and try out different regions with quick feedback. If you have a beefy machine you can keep it all in RAM sometimes and it works better.. but it's not ideal (my 16GB machine can only handle simpler scenarios)
I'll take a closer look at in the future. At the moment I just use Java's default TIFF reader and ImageBuffer class. Maybe it'd be easy to convert to COG format and adapt in a COG reader
Almost any digital camera RAW format is TIFF inside. And you can see how much kludges good metadata library needs to read all of them: offsets from the IFD, offsets from beginning of file with or without header, offsets from fields in IFD, etc, etc, etc. You take TIFF, you change header to make your format, and then you cannot implement this TIFF properly!
Even DNG (which is tiff inside) is mangled by camera firmware authors!
And the (early) availability of well made library, LibTIFF by Sam Leffler.
I used it extensively from 1995 on, but only found out that according to Wikipedia is dates back to 1988!
I had exposure to TIFF files shortly after the format creation in 1985/86, before the final form specification in 1992.
Not mentioned in either the article or the tail end wikipedia article iamge was the early adoption of TIFF by the mapping and geodetic community to store raster line data (maps, images, and raw sat and instrument platform multichannel line data).
The tagging format made the embedding of spheroids, datums, projections, origins, lens and focal specifications relatively easy (plus or minus the usual Tower of Babel Tag Naming and Meaning Confusion).
I have a habit of filling my head with all kinds of trivia/information. Sometimes I think this is a useless habit.
Not today. I will try to remember the name of Mr. Stephen Carlsen as the inventor of TIFF format as long as I can. As a mediocre programmer, it is the least amount of respect I can pay for an unsung but talented engineer of an era that is fast going past us.
The article is great but the web site is supposedly related to a book "inventing the future".. which is nowhere to be found. Other than a big, slowly loading graphic, 3 posts and indexes for the book... the site doesn't provide a clue about where to acquire the actual (PDF only?) book.
I assume you have to sign up to find out more ?
On the web I can only find articles about the book.
So.. what is the deal in making the actual book hard to find ?
I had a similar issue, clicking the author's name gets you to a decent page, but yeah I'd actually prefer if he made it a bit easier to buy the book! I'll have to get it now after such a nice article
This page [0] still links to (when clicking the main image) an oddish seeming corner of your site that mentions purchasing the book but has no link to it
Based on the quality of the article, the subject matter of the book being right in the center of my wheelhouse and the references I could find on the internet, I just ordered a copy (apparently a paper copy), look forward reading it.
TIFF indeed -- I recall the floppy disk for Mac mailed from Seattle with the TIFF spec printed on paper. A few weeks later, another graphics editor with TIFF support. I never, ever heard the name Carlsen until today. Thank you for this article
the article shows scans of the research reports listing only Carlsen as their author, you could have just linked to one in the first paragraph of the Wikipedia page to support his sole inventorhood, right
per: the scans, the patent isn't for TIFF and the articles misspell his name. I think that the Paul Brainerd (Aldus cofounder) interview with the Computer History Museum, when he names Stephen, will be sufficient for Wiki.
I participated in creating a history book, in regard to an organization in which I’m involved.
It took eight years, and was a lot of work. The process that he mentioned is quite familiar. Many of the folks we interviewed have since passed away. Some, before the book was complete.
Threads like this are, in a sense, like a digital wake - we can all mourn Carlsen a little bit now. I remember the first time I saw a .TIF - on Rainbow Paint, a free paint program bundled with a Dexxa mouse my old man bought as an upgrade for our 286. To me, running across a random .tif somewhere was such a delight, something I could open as a surprise, maybe re-use parts of, zoom into, etc. or share on a disk with a friend. It's quaint, now...
I had downloaded the final Aldus TIFF specifications document, hoping to find the author’s name. However, the name is seemingly written in white text on white paper - making it invisible. What?
Is there an explanation for this that I missed? Was it an Easter Egg left by the author?
Just as a side note, there are two versions of tiff6.pdf (titled “TIFF, Revision 6.0, Final — June 3, 1992”) on the ’net: one[1] that mentions Aldus on the title page and one[2] that mentions Adobe. Only the Aldus one has the invisitext. (Curiously, the metadata says it’s newer.)
[2] SHA256: 8cb1e1a2226e423ba8b88f57366a30ef1b7ad6109443ebdda072b952739a8d76, modify date 1995-09-14, created by Acrobat Distiller 2.1 for Power Macintosh, e.g. https://download.osgeo.org/libtiff/doc/TIFF6.pdf
I've found a number of hidden items in PDFs masked in the same way. The one that comes to mind immediately are Dell product code names hiding in official spec PDFs. (The PowerConnect 6200-series switches are "Kinnick", for example.) I always assumed it was a lazy redaction method and people weren't necessarily aware the text wasn't actually redacted.
Respect to those unsung engineers who made such lasting contributions, and to the author as well. This kind of work is not easy, but truly meaningful.
I do have a question, though: shouldn’t the creation of industry standards also allow individual attribution, similar to how patents credit inventors?
Crazy this information would have probably been lost in time if one single person on this planet didn’t give a shit like the rest of us.
What a journey and congratulations to SC (don't want to spoil it) on your 15 minutes and rightful restoration as inventor of TIFF, take your place in history.
thanks righthand, i guess it was just curiosity that led me down the path. most people do give a sh## but i hear you. i also had the time to search, as i wasn't super busy with work.
Thank you so much again for your efforts, I sound brash but this really is inspiring and as you demonstrated and have indicated, we can all use our free time to easily make the world a little more accurate and better.
I'd like to add, it's really nice when good people get recognized purely for their work, rather than because they're loud enough to be noticed. I very much appreciate people like yourself that take the time to look for the quiet ones like this.
Did a similar deep dive for one of the posters for the cult classic movie Possession (1981). Just giving random phone numbers a call is incredibly effective, lots of people are happy to reminisce about old work and have great stories.
Very often these people are so humble and so amazed to find that anyone cares so much about some little project they did. I've brought some people to comic cons and they have been blown away by the fans they never knew they had. (and they always have fascinating industry stories to tell)
It’s so inspiring to see someone spend years uncovering the real people behind tech we use every day. This kind of dedication keeps our digital history alive.
Glad that the information was preserved in the magazines, usenet messages and just text files. That will not happen with the modern web software, the internet is the dark ages of our time. All those Java,Flash amazing pieces of software and the stories of their creators will be gone long before the internet dies from LLM slop.
And Geocities, Vine, Google+, Anglefire, Tripod, Xoom, Homestead, Lycos communities, AOL Hometown, MSN Groups, 50megs.com, etc, etc.... not to mention small specialty sites like em411.com. All that content/history, just poof.
They found the Vine archives recently. Doesn't mean they'll get uploaded as Musk wants the new Vine to just be AI waifus. But the files still exist on a disk or tape somewhere.
I just remembered Orkut. Though I suspect Google has backups or Orkut and Google+ somewhere. I wonder if Yahoo Answers is still on a tape somewhere?
Back when I was working in the book publishing industry (writing and typesetting computer books using Quark XPress [1] on the old MacOS [version 7, 8 and 9 IIRC], before Adobe's InDesign [2] ruled the earth), TIFF was all the rage. Probably still is.
I think the reason TIFF was so prevalent was it already had support for CMYK color space (even though many books were printed in black and white) and for lossless compression (as TFA mentions).
It was a "one size fits all" format and so our 100 or 250 MB (!) Zip drives [3] exchanged between authors/publisher/typesetters often contained TIFF files.
> For as long as I have published my books, one of my overarching goals was to give credit to those who actually invented the hardware and software that we use.
John, your 10,000+ hours of research isn't just documenting inventions—it's lighting up the lamps for those hidden geniuses. Stephen Carlsen's story reminds me that our digital world stands on the shoulders of countless 'Mr TIFF's. RIP, and thanks for making his name eternal. PS: What's your next invention-hunting target?
i have concentrated on apple in the era when jobs isn't there to cancel or squash ideas and invention. many folks, not unlike stephen, come out of the woodwork. and i have slowly connected with them.
last week i documented (through interviews) tom gilley's work on apple's first hand-held pen-driven tablet called scribe. this included innovation and invention of suppressing electro mmagnetic interference, removng contemporary tech's weight, stretching battery output and so forth - as well as writing a slimmed down version of the mac/os and handwriting recognition rather than writing one completely from scratch (which was what newton/dylan) were doing.
Please do not let my comment take away your enjoyment of the article.
I hate to nit-pick on such a beautiful story but that it ended with a faux-Ghibli profile picture is just sad.
How can someone working so hard to humanize technology and preserve history, justify this soul-less commodification of art? Do the animators deserve to get treated as anonymous model trainers without their consent, names and frames lost in a dead ocean of bit-vectors?
Don't have much to add except to mention again that the magic number for TIF is 42, and it's 42 because of the meaning of 42:
https://web.archive.org/web/20210108174645/https://www.adobe...
And here is the author himself confirming that in the Wikipedia talk page for TIFF! https://en.wikipedia.org/wiki/Talk:TIFF/Archive_1#h-Source_f...
Great find! And oh no, it’s complete with the customary blissfully unaware user replying to say he’s wrong!
Hindsight is 20/20 and I loved TFA and I don't want to ruin it but... that comment was there from 2007 and the Wikipedia user bio was pretty clear since the beginning (https://en.wikipedia.org/w/index.php?title=User:Scarlsen&old...)
Yes but it's not easy to find random Wikipedia user pages, or even find the specific talk page comment in the archives without knowing what to look for. Go ahead, find a friend, give them no clues, and see if they find it.
Yes but the author was specifically investing over this, had a clue on a similar name and even edited the same page himself.
I don't know what “investing over” means, but all I can say is (repeating myself): try it with a friend, without giving them the benefit of hindsight.
(Very few visitors to a Wikipedia page read its talk page, very few of them will further look at the archives of the talk page, let alone read every single comment and its corresponding commenter's name, and in this case as soon as the author knew the spelling to look for, the rest was straightforward for them.)
well said svat. without laboring the point, "once you know - you know". until then it is like trying to find a needle in a haystack - and some people do not want to be found.
I mean that the author was literally writing a book about this and literally conducting an investigation about who was the real author of the TIFF spec. He interviewed other people about it, he had a name that had a typo in it, he even found a white-on-white line in a PDF with the real name. He was clearly putting effort into the research.
We are humans, everybody can miss things, I mentioned "hindsight is 20/20" but still, it was in the Wikipedia discussion page for the TIFF article all the time. It's a matter of fact and some random HNer found it in minutes/hours.
I repeat myself, it was probably better that he didn't found that out and went to write a hand-written letter to the alleged author's home address, it created a much deeper human bond, which is especially meaningful since Stephen Carlsen passed away not much later.
thanks darkwater
And interestingly, the person he replies to is taviso [0][1]
[0] https://news.ycombinator.com/user?id=taviso
[1] https://en.wikipedia.org/wiki/Tavis_Ormandy
(Also: 42 is the answer to everything because it's the ascii code for *).
Was that a happy coincidence or intentional?
Perfectly pure happiness:
"The answer to this is very simple. It was a joke. It had to be a number, an ordinary, smallish number, and I chose that one. Binary representations, base thirteen, Tibetan monks are all complete nonsense. I sat at my desk, stared into the garden and thought '42 will do' I typed it out. End of story." from the man himself[1]
...but let us not ruin a good story with the truth. Remember why earth was built. The "real" answer might then be flowing in the ether.
[1] https://en.wikipedia.org/wiki/Phrases_from_The_Hitchhiker%27...
I think it was intentional but I don't have a source.
Edit: from the other comment, it appears it was in fact random...
> the ASCII code for h
Umm. The ASCII code for h is 102 ;)
I don't know why or how you're seeing an h? I'm talking about the asterisk.
I could be wrong but the winky face leads me to believe theyre referencing the hunter2 password meme
Ah, thanks! I wasn't familiar with this meme. It is funny.
Are talk pages accepted as a source for the same article?
Talk pages aren’t valid sources in general. In this case the author is dead and an established expert having published in the field, so I guess it’s fine, but I wouldn’t bet.
https://en.wikipedia.org/wiki/Wikipedia:Verifiability#Self-p...
42 even shows up in the late mr. Carlsen's obituary. [1]
[1]https://www.mountainviewtacoma.com/obituaries/stephen-carlse...
I remember Steve.
42 is an extremely non-special number. Does anyone know if it appeared in the CS field before Douglas Addams "invented" it?
What do you mean, non-special? 42 is the magic constant of the smallest non-trivial magic cube.
It’s also an integer. Who ch I pretty special when you consider that most numbers in nature are real.
I have yet to see a number in nature.
Nature is full of them.
But they're real, right? Or is too complex to just say so?
There isn’t a fraction of me which didn’t appreciate that comment
*was
Based on the same algorithm as https://xkcd.com/221/
If you had told me an article ostensibly about a file format would have me teary-eyed by the end I wouldn't have believed you. This is beautiful, thank you!
Thanks oisin, it's a beautiful story and his ex-wife gave me permission to share.
A huge number of us labor behind the scenes with no public acknowledgment or credit. For each idea brought to life we might hope for -at most- an epitaph carved in expired patent claims.
This story is touching.
thanks daniel. i swapped a few emails with stephen's ex-wife and they melted my heart. i asked if she was OK with sharing this story of a humble genius with the wider world and she said, yes.
It is, indeed. Thanks for writing that up, it touched me.
And thanks for being the historian of our culture-that-eschews culture (or so it seems to me sometimes, that tech tries to exist in a perennial present without acknowledging it's roots and history)
thanks isoprophlex for the acknowledgment. so many of the folks that i have interviewed, contributed so much to making tech or a product better for all. especially at or around apple. they receieved none of the credit that the 'names' got. ;-) you know who i mean.
Thank you both!
I checked the TIFF talk page and found comments from:
https://en.wikipedia.org/wiki/User:Scarlsen
Turns out the answer was on Wikipedia already :).
Thanks! If you look at his (logged-in) edits on Wikipedia (https://en.wikipedia.org/wiki/Special:Contributions/Scarlsen ), then apart from the lone comment on the talk page (about the reason for "42") and creating that user page, he has two edits to the TIFF article:
- one of them clarifies the (non-)involvement of Microsoft: https://en.wikipedia.org/w/index.php?title=TIFF&diff=prev&ol...
- and the other is even more interesting: though he is being scrupulous and removing a sentence that has no published citations, in his edit summary he confirms that it is basically true:
> The author of the original TIFF specification wanted TIFF to stand for "The Image File Format", but he was overruled by Aldus' president Paul Brainerd on the grounds that it sounded presumptuous.
(The edit summary says: Removed the "The Image File Format" sentence, since it only has eye-witness support (me, for one), but no published citatations)
Ok so then we could technically edit it back in since he's a primary source, right?
It's been a few years since I edited Wikipedia seriously, but the criterion for inclusion on Wikipedia is/was “verifiability, not truth” (https://en.wikipedia.org/wiki/Wikipedia:Verifiability,_not_t...) – what matters is not whether something is true, but whether it has been published in a reliable source (https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources). Accordingly, Wikipedia tries to be based on secondary sources (rather than primary and tertiary ones). The relevant section (https://en.wikipedia.org/wiki/Wikipedia:No_original_research...) says, among other things:
> Primary sources that have been reputably published may be used in Wikipedia, but only with care
and I imagine a Wikipedia edit summary does not count as a reliable source. (For one thing, despite it being very plausible that the Wikipedia user Scarlsen who signed himself as Stephen E Carlsen is indeed that person—I believe it completely!—it cannot be guaranteed that it wasn't an impostor, for example.)
That would be what Wikipedia calls "original research". A big no-no on wikipedia. At a minimum he would have to tweet or blog about it and link the tweet or blog. And even then that's a primary source, which wikipedia considers less valuable. Ideally he would get someone else to report on his tweet/blog and use that as source. Then the wikipedia gods are happy
https://en.wikipedia.org/wiki/Wikipedia:No_original_research
So can we use this conversation on HN as a secondary source, and edit the deletion back in citing Hacker News?
Technically yes, but I'm fairly sure Wikipedia wants cited sources, not "I'm the guy, I said so" anecdotal sources.
Of course, if he was still alive he could have written a blog post or something like that and use that as a source, much like how it's likely this blog post will be used as a source for things surrounding the format and person.
His lone comment:
>Yes it is true: the second word of a TIFF file, 42, was indeed taken from the answer to the Ultimate Question of Life, the Universe, and Everything, from Hitchhikers_guide_to_the_galaxy. StephenECarlsen 23:38, 12 October 2007 (UTC)
If anyone can contact John Buck this sounds like information he'd be interested in. Also an interesting avenue for future investigative work.
thanks adam
Hey John, I'm just curious how people find these comments about "would be nice if X saw this" on HN. I don't think there's any pinging behavior. Did somebody message you? Did you just happen to read it? Do you have an eldritch curse that summons you when called by name?
Somebody subscribed to my blog with ref to hacker news so i just poked my head in :-)
I use f5bot.com, which monitors HN, Reddit, and Lobsters. (I have no connection to f5bot except being a happy user.)
Not who you're asking for, but generally I think it's just a case of the author also being an HN regular. Although, I suppose you could set up some Google Alerts for mentions of your blog posts.
Once upon a time there was a guy who went by Kibo who would search Usenet feeds for posts mentioning his username and reply to them...
I bet Musk hacked something together (or has a column in TweetDeck if that's still around) that continuously searches for mentions. I wonder if there's a tool like that that covers more of the internet, although the primary users would only be famous people and/or their agents / social media staff.
There are lots of tools that do this if you are willing to pay for it. "Social Media Monitoring" or "Brand Monitoring" would be the keywords to search for
You can try our tool for example: https://KWatch.io
It monitors keywords mentioned on X, Linkedin, Facebook, Youtube, Quora, Reddit, and YC.
Kibo greps
From what I've seen, sometimes people see that their blog post got a lot of referral traffic from Hacker News and they come have a look to see where it came from.
I've seen Ken Shirriff do this. You mention him and suddenly he's there.
hi! :-) As I commented elsewhere, f5bot.com is a nice, free tool to monitor HN
The great thing about TIF was it's extensibility. Flexible (data could be stored as tiles or in stripes), multiple compression options etc.
Well documented spec, easy to bolt on extras either as public tags - GeoTIFFs added projection metadata - or private, for your own needs.
Back in the day, to improve a desktop application's performance I found it was simple to create a custom reader and writer to handle cases where tiles were completely one single colour removing the need to decompress at run time.
Thank you TIFf!
Perhaps the greatest thing about TIFF, but also the most horrible things, and probably why TIFF is mostly historical. It was so extendable that no two programs ever accepted the exact same TIFF extensions. (omitting the war story)
edit: forgot about byte order...
But most of these variations were part of the spec (endianness with II or MM, later magic 43 for bigTIFF 64bit extension). I work with tiff and tiff-derived formats in digital microscopy where its very much not historical. And the alternatives (DICOM supp 145, vendor-specific garbage ... and thats it) are worse.
I quite like the format, the only thing I would change is to have the option not to store directory information in a linked list spread throughout the file but in a simple array. Duplicate it at the beginning and end of the file and you've got resilience too (important in the age of floppies)
We have hundreds of thousands of TIFF files where I work which are scans of questionnaires filled out by clinical trial participants. The one annoying thing is that web browsers don’t natively display them. I did some incredibly inefficient JavaScript bs to decode the pixel data, plop it in a canvas, get a PNG data url from it, and set that as the src for an img element xD (why not just display the canvas? because I was too lazy to manually handle resizes…) good times
TIFF is still very much alive in certain circles, see for example https://cogeo.org/ The format is basically a TIFF file with attached georeferencing information and with the data organized by geographical sector, enabling fast downloads of regional subsets.
The Adobe DNG standard for raw camera images is based on TIFF as well. DNG is used in lots of places, including the raw capture support built into all modern iOS and Android smartphones.
I’ve been using both TIFF and DNG this very week in my work (https://filmlabapp.com), so I was happy to read this post and learn about Steve Carlsen aka Mr. TIFF, whose work we’re still building on 39 years later.
Not only DNG, but I think most (maybe all?) raw camera files are based on TIFF.
The top 3 (Canon, Sony, Nikon) are all based on TIFF, and that accounts for the majority of the market share. Search for "TIFF-based" here:
https://exiftool.org/#supported
Not the .IA files from old Sinar digital backs! Those are based on DOOM PWADs (lol). But otherwise mostly yes this is true for nearly every other format as far as I am aware.
I'm not super knowledgeable about this stuff.. but out of curiosity, what advantage does it have over NetCDF?
I wrote a program processing GeoTIFF data. When I had started this project I chose GeoTIFF mostly b/c i wanted something simple. And I could load them in to Java's BufferImage class and manipulate them that way. But it seems all the pros exclusively use NetCDF and GeoTiffs are for noobs (working with atmospheric science data here)
GeoTIFF does extend "images" to cover more usecases, but a lot of stuff doesn't fit (like say a wind vector) and then you need some other container or metadata b/c you generally have many images. So I get the sense the complexity just ends up being moved elsewhere.
Generally speaking I would classify TIFF and its variants as imaging formats (or for very simple numerical datasets), and NetCDF as more suited for raw data, in particular multi-dimensional data with time series, etc.
For forecast and climatological data I find NetCDF is vastly superior, but also much more complicated to work with due to the capabilities and how open the format is. Just have a look at the complexity of the CF Conventions to see what I mean: https://cfconventions.org/cf-conventions/release/v1.12.0/cf-...
For visualizing orthophotos and the like, I would choose GeoTIFF any day of the week, as they're easy to visualize across platforms using existing libraries. Using COGs you also get the functionality of a spatial index within each GeoTIFF file, meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request.
Yeah the complexity of NetCDF was the ick factor that made me use GeoTIFF. It's just not the level of complexity I wanted to deal with in-program. I didn't want to wade through the details to understand the format, and I didn't want to then lock users to some subset
Seemed easier to let users preprocess their NetCDFs into GeoTIFFs manually. I have a bunch of hacky scripts to massage NetCDFs from different sources in to compatible GeoTIFFs
> meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request.
Interesting. My performance bottleneck right now is the user selects a small regions and then the program has to read in GBs of global precipitation maps (from IMERG) and cut out tiny squares. In the extreme cases it can mean ~2 minutes of waiting for a result. This means the user can't casually select and try out different regions with quick feedback. If you have a beefy machine you can keep it all in RAM sometimes and it works better.. but it's not ideal (my 16GB machine can only handle simpler scenarios)
I'll take a closer look at in the future. At the moment I just use Java's default TIFF reader and ImageBuffer class. Maybe it'd be easy to convert to COG format and adapt in a COG reader
This is why some people consider TIFF to stand for “Thousands of Incompatible File Formats”.
Unfortunately, it doesn't help.
Almost any digital camera RAW format is TIFF inside. And you can see how much kludges good metadata library needs to read all of them: offsets from the IFD, offsets from beginning of file with or without header, offsets from fields in IFD, etc, etc, etc. You take TIFF, you change header to make your format, and then you cannot implement this TIFF properly!
Even DNG (which is tiff inside) is mangled by camera firmware authors!
And the (early) availability of well made library, LibTIFF by Sam Leffler. I used it extensively from 1995 on, but only found out that according to Wikipedia is dates back to 1988!
Carlsen's TIFF extensibility was what Milne and Lentczner used as a guiding principal when building AIFF, as did Leak on QuickTime years later.
His obituary is lovely: https://www.mountainviewtacoma.com/obituaries/stephen-carlse...
Sillepses are always great.
I had exposure to TIFF files shortly after the format creation in 1985/86, before the final form specification in 1992.
Not mentioned in either the article or the tail end wikipedia article iamge was the early adoption of TIFF by the mapping and geodetic community to store raster line data (maps, images, and raw sat and instrument platform multichannel line data).
The tagging format made the embedding of spheroids, datums, projections, origins, lens and focal specifications relatively easy (plus or minus the usual Tower of Babel Tag Naming and Meaning Confusion).
I have a habit of filling my head with all kinds of trivia/information. Sometimes I think this is a useless habit.
Not today. I will try to remember the name of Mr. Stephen Carlsen as the inventor of TIFF format as long as I can. As a mediocre programmer, it is the least amount of respect I can pay for an unsung but talented engineer of an era that is fast going past us.
Am I missing something ?
The article is great but the web site is supposedly related to a book "inventing the future".. which is nowhere to be found. Other than a big, slowly loading graphic, 3 posts and indexes for the book... the site doesn't provide a clue about where to acquire the actual (PDF only?) book.
I assume you have to sign up to find out more ?
On the web I can only find articles about the book.
So.. what is the deal in making the actual book hard to find ?
Edit: I think I cracked the code: Click Home, Open "Close Your Rings" article, scroll all the way down, find link: https://books.by/john-buck?ref=inventingthefuture.ghost.io
I had a similar issue, clicking the author's name gets you to a decent page, but yeah I'd actually prefer if he made it a bit easier to buy the book! I'll have to get it now after such a nice article
i didn't want to push the book too hard, given the tone of the story. thanks for the feedback.
That makes perfect sense, you're a gent.
This page [0] still links to (when clicking the main image) an oddish seeming corner of your site that mentions purchasing the book but has no link to it
[0] https://inventingthefuture.ghost.io/
hi andre, thanks for the feedback. there is a url link within the article to the book which uses a new self publishing method called books.by
Ah, I see, okay.
Based on the quality of the article, the subject matter of the book being right in the center of my wheelhouse and the references I could find on the internet, I just ordered a copy (apparently a paper copy), look forward reading it.
TIFF indeed -- I recall the floppy disk for Mac mailed from Seattle with the TIFF spec printed on paper. A few weeks later, another graphics editor with TIFF support. I never, ever heard the name Carlsen until today. Thank you for this article
thanks gnerd
Quietly thankful that the spec author didn't proclaim that we've been mispronouncing "TIFF" all these years.
The "TI" is really "Tl" and is a voiceless alveolar lateral affricate.
TIL:
* https://en.wikipedia.org/wiki/Voiceless_alveolar_lateral_aff...
I tried to pronounce it, but summoned a daemon instead
RIP Mr. TIFF. Hoping we continue to document these incredible engineers and their work before it's lost to the sands of time/pits of LLM muck.
i've interviewed 100 folks in this space, in part because they are older than us.
Beautiful essay. So much of the tech we use today originates from quiet humble builders and creators like Mr TIFF.
thanks for the response burnto
:) Pleased to see the wikipedia change landed without drama. It’s still there as of writing.
i crossed my fingers for the first 24 hours but i now think admins and mod, to their credit, understand it's the truth.
the article shows scans of the research reports listing only Carlsen as their author, you could have just linked to one in the first paragraph of the Wikipedia page to support his sole inventorhood, right
per: the scans, the patent isn't for TIFF and the articles misspell his name. I think that the Paul Brainerd (Aldus cofounder) interview with the Computer History Museum, when he names Stephen, will be sufficient for Wiki.
Pretty amazing investigation work. Very nice to see that credit is being given where due.
I participated in creating a history book, in regard to an organization in which I’m involved.
It took eight years, and was a lot of work. The process that he mentioned is quite familiar. Many of the folks we interviewed have since passed away. Some, before the book was complete.
Threads like this are, in a sense, like a digital wake - we can all mourn Carlsen a little bit now. I remember the first time I saw a .TIF - on Rainbow Paint, a free paint program bundled with a Dexxa mouse my old man bought as an upgrade for our 286. To me, running across a random .tif somewhere was such a delight, something I could open as a surprise, maybe re-use parts of, zoom into, etc. or share on a disk with a friend. It's quaint, now...
I'll admit that the only time I ever interacted with a TIFF file was the buffer overflow exploit in the PSP Photo Viewer. RIP Mr. TIFF.
https://en.wikibooks.org/wiki/PSP/Homebrew_History?useskin=v...
Just as a side note, there are two versions of tiff6.pdf (titled “TIFF, Revision 6.0, Final — June 3, 1992”) on the ’net: one[1] that mentions Aldus on the title page and one[2] that mentions Adobe. Only the Aldus one has the invisitext. (Curiously, the metadata says it’s newer.)
[1] SHA256: dbcdf729182937ecff415dfd06806894bf03bfd741291aa3ad7ba45335673def, modify date 2002-05-10, created by Acrobat Distiller 4.05 for Windows, e.g. https://www.itu.int/itudoc/itu-t/com16/tiff-fx/docs/tiff6.pd...
[2] SHA256: 8cb1e1a2226e423ba8b88f57366a30ef1b7ad6109443ebdda072b952739a8d76, modify date 1995-09-14, created by Acrobat Distiller 2.1 for Power Macintosh, e.g. https://download.osgeo.org/libtiff/doc/TIFF6.pdf
I've found a number of hidden items in PDFs masked in the same way. The one that comes to mind immediately are Dell product code names hiding in official spec PDFs. (The PowerConnect 6200-series switches are "Kinnick", for example.) I always assumed it was a lazy redaction method and people weren't necessarily aware the text wasn't actually redacted.
Depends on what format you view it in. For the printed version of the spec, it's sufficient ;)
Respect to those unsung engineers who made such lasting contributions, and to the author as well. This kind of work is not easy, but truly meaningful. I do have a question, though: shouldn’t the creation of industry standards also allow individual attribution, similar to how patents credit inventors?
Beautiful and moving. Thank you author of the article and thank you Mr TIFF
I was not expecting the emotional ending. Really well done.
thanks upvoter
Crazy this information would have probably been lost in time if one single person on this planet didn’t give a shit like the rest of us.
What a journey and congratulations to SC (don't want to spoil it) on your 15 minutes and rightful restoration as inventor of TIFF, take your place in history.
thanks righthand, i guess it was just curiosity that led me down the path. most people do give a sh## but i hear you. i also had the time to search, as i wasn't super busy with work.
Thank you so much again for your efforts, I sound brash but this really is inspiring and as you demonstrated and have indicated, we can all use our free time to easily make the world a little more accurate and better.
I'd like to add, it's really nice when good people get recognized purely for their work, rather than because they're loud enough to be noticed. I very much appreciate people like yourself that take the time to look for the quiet ones like this.
Oh man, this is a really nice ending to his story. I'm glad this happened.
thanks for the feedback
Did a similar deep dive for one of the posters for the cult classic movie Possession (1981). Just giving random phone numbers a call is incredibly effective, lots of people are happy to reminisce about old work and have great stories.
Very often these people are so humble and so amazed to find that anyone cares so much about some little project they did. I've brought some people to comic cons and they have been blown away by the fans they never knew they had. (and they always have fascinating industry stories to tell)
눈물나게 감동적이었습니다.
Translation: “It was so moving that I cried.”
It’s so inspiring to see someone spend years uncovering the real people behind tech we use every day. This kind of dedication keeps our digital history alive.
thanks for the positive vibes scriptweaver
Glad that the information was preserved in the magazines, usenet messages and just text files. That will not happen with the modern web software, the internet is the dark ages of our time. All those Java,Flash amazing pieces of software and the stories of their creators will be gone long before the internet dies from LLM slop.
I think of all the content we've lost already. MySpace files are lost. Friendster archives are gone. So many YouTube videos lost to time.
I link to a lot of stuff on my personal website and every month i check the links. About a dozen or so are dead every month, many on YouTube too.
I now adopted the practice of recovering the texts I deem worthy from way back machine and downloading all yt videos I really like locally.
But ofc one day I’ll also hit the bucket; still have to work out a contingency plan for my archive for that …
And Geocities, Vine, Google+, Anglefire, Tripod, Xoom, Homestead, Lycos communities, AOL Hometown, MSN Groups, 50megs.com, etc, etc.... not to mention small specialty sites like em411.com. All that content/history, just poof.
They found the Vine archives recently. Doesn't mean they'll get uploaded as Musk wants the new Vine to just be AI waifus. But the files still exist on a disk or tape somewhere.
I just remembered Orkut. Though I suspect Google has backups or Orkut and Google+ somewhere. I wonder if Yahoo Answers is still on a tape somewhere?
This is valuable work in cataloging the foundations of the computing industry!
It's weird to see times one has lived through presented as ancient history....
Computer science is such a young field that we can still sit at the feet of the giants whose shoulders we stand on.
Thank you John Buck for this article, it is so interesting to read how something so common was invented. RIP Mr Tiff
thank you
Thanks for researching this, I updated the french wiki page based on your work
thanks for the positive vibes yann, that's a nice touch
Nice read. It was touching and it's great that credit is given where credit is due now.
Me: “This link can’t possibly be about what I think it might be about.” Me, seconds later: “Yes it is!!”
And that’s a wonderful lesson to try searching alternate spellings of names for an oral history.
I see you played the NYT Connections on 11/4.
Beautiful
thank you sethx
Back when I was working in the book publishing industry (writing and typesetting computer books using Quark XPress [1] on the old MacOS [version 7, 8 and 9 IIRC], before Adobe's InDesign [2] ruled the earth), TIFF was all the rage. Probably still is.
I think the reason TIFF was so prevalent was it already had support for CMYK color space (even though many books were printed in black and white) and for lossless compression (as TFA mentions).
It was a "one size fits all" format and so our 100 or 250 MB (!) Zip drives [3] exchanged between authors/publisher/typesetters often contained TIFF files.
> For as long as I have published my books, one of my overarching goals was to give credit to those who actually invented the hardware and software that we use.
So thank you Mr. Stephen "TIFF" Carlsen!
[1] https://en.wikipedia.org/wiki/QuarkXPress
[2] https://en.wikipedia.org/wiki/Adobe_InDesign
[3] https://en.wikipedia.org/wiki/Zip_drive
John, your 10,000+ hours of research isn't just documenting inventions—it's lighting up the lamps for those hidden geniuses. Stephen Carlsen's story reminds me that our digital world stands on the shoulders of countless 'Mr TIFF's. RIP, and thanks for making his name eternal. PS: What's your next invention-hunting target?
hi kilibe,
thanks for your interest.
i have concentrated on apple in the era when jobs isn't there to cancel or squash ideas and invention. many folks, not unlike stephen, come out of the woodwork. and i have slowly connected with them.
last week i documented (through interviews) tom gilley's work on apple's first hand-held pen-driven tablet called scribe. this included innovation and invention of suppressing electro mmagnetic interference, removng contemporary tech's weight, stretching battery output and so forth - as well as writing a slimmed down version of the mac/os and handwriting recognition rather than writing one completely from scratch (which was what newton/dylan) were doing.
so there's a lot of conversations to document.
i do like the 'lighting up the lamps'
[dead]
[dead]
[dead]
Please do not let my comment take away your enjoyment of the article.
I hate to nit-pick on such a beautiful story but that it ended with a faux-Ghibli profile picture is just sad.
How can someone working so hard to humanize technology and preserve history, justify this soul-less commodification of art? Do the animators deserve to get treated as anonymous model trainers without their consent, names and frames lost in a dead ocean of bit-vectors?
my kids made the avatar so...sorry if it triggers
I understand, and I apologize for the rant.
Thank you for all the efforts that went into preserving the memories of those that built the world around us.