> Perhaps you’re a user of LLMs. I get it, they’re neat tools. They’re useful for certain kinds of learning. But I might suggest resisting the temptation to use them for projects like this. Knowledge is not supposed to be fed to you on a plate
Am I the only one using LLMs as if they were a search engine? So before LLMs I was searching on Google things like "pros cons mysql mongodb". I would read the official documentation of each db, forums, blog posts, stackoverflow entries, etc. It was time consuming on the searching side. The time it took to read all the sources was fine for me (it's learning time, so that's always welcomed). Now with LLMs, I simply prompt the same with a little bit more of context "pros and cons of using mysql vs mongodb when storing photos. Link references". So, I get a quick overview of what to keep an eye on, and the references are there to avoid relying on hallucination.
It's true that sometimes I go ahead and say "give me a data schema for storing photos metadata in postgres. I wanna keep X in a different table, though" (or something like that). But I do that because I know very well what the output should look like (I just don't wanna spend time typing it, and sometimes I forget the actual type that I should use (int vs integer?)).
The few times I've used LLMs as question answering engines for anything moderately technical, they've given subtly-but-in-important-ways incorrect information such that taking them at face value would've likely lost me hours or days of pursuing something unworkable, even when I ask for references. Whether or not the "references" actually contain the information I'm asking for or merely something tangentially related has been rather hit or miss too.
The one thing they've consistently nailed has been tip-of-my-tongue style "reverse search" where I can describe a concept in sufficient detail that they can tell me the search term to look it up with.
Absolutely. And I’m finding the same with “agent” coding tools. With the ever increasing hype around Cursor I tried to give it a go this week. The first 5 minutes were impressive, when I sent a small trial ballon for a simple change.
But when asking for a full feature, I lost a full day trying to get it to stop chasing its tail. I’m still in the “pro” free trial period so it was using a frontier model.
This was for a Phoenix / Elixir project; which I realize is not as robustly in the training data as other languages and frameworks, but it was supposedly consuming the documentation, other reference code I’d linked in, and I’d connected the Tidewave MCP.
Regardless, in the morning with fresh eyes and a fresh cup of coffee, I reverted all the cursor changes and implemented the code myself in a couple hours.
>The one thing they've consistently nailed has been tip-of-my-tongue style "reverse search" where I can describe a concept in sufficient detail that they can tell me the search term to look it up with.
This is basically the only thing I use it for. It's great at it, especially given that Google is so terrible these days that a search describing what you're trying to recall gets nothing. Especially if it involves a phrase heavily associated with other things.
For example "What episode of <X show> did <Y thing> happen?" In the past, Google would usually pull it up (often from reddit discussion), but now it just shows me tons of generic results about the show.
Yes, you have to be very careful when querying LLM's, you have to assume that they are giving you sort of the average answer to a question. I find them very good at sort of telling me how people commonly solve a problem. I'm lucky, in that the space I've been working has had a lot of good forums training data, and the average solution tends to be on the more correct side. But you still have to validate nearly everything it tells you. It's also funny to watch the tokenization "fails". When you ask about things like register names, and you can see it choose nonexisting tokens. Atmel libraries have a lot of things like this in them
And the output will be almost correct code, but instead of an answer being:
PORT_PA17A_EIC_EXTINT1
you'll get:
PORT_PA17A_EIC_EXTINT_NUM
and you can tell that it diverged trying to use similar tokens, and since _ follows EXTINT sometimes, it's a "valid" token to try, and now that it's EXTINT_ now NUM is the most likely thing to follow.
That said, it's massively sped up the project I'm working on, especially since Microchip effectively shut down the forums that chatgpt was trained on.
This. I was skeptical at first, but it is indeed good at searching and answering questions without! That said, I still have to double-check results for niche queries or about stuff that is relatively new. Sometimes, the "sources" for the answers are just someone's opinions — unsubstantiated by any facts — on an old Reddit post that's only tangentially related to the topic. And sometimes, you simply know that manual search and digging through SO answers yourself will yield better results. At this point I've developed a gut feeling that helps me decide whether to prompt Perplexity or just g**gle it.
> the "sources" for the answers are just someone's opinions — unsubstantiated by any facts
Isn't that the nature of the web?
I mean, that's exactly what I expect from web searches, so as long as you don't consider the fancy-looking [1] citations "scientific", it just digs up the same information, but summarized.
I don't expect miracles: Perplexity does the same things I've been doing, just faster. It's like a bicycle I guess.
I agree. Use with caution. One of my personal pet peeves with LLM answers is their propensity to give authoritative or definite answers, when in fact they are best guesses, and sometimes pure fantasy.
> Now with LLMs, I simply prompt the same with a little bit more of context "pros and cons of using mysql vs mongodb when storing photos. Link references".
In near future, companies will probably be able to pay lots of money to have their products come up better in the comparison. LLMs are smart enough to make the result seem "organic" -- all verifiable information will be true and supported by references, it will only be about proper framing and emphasis, etc.
I'd say LLMs have helped a lot with this problem actually, by somehow circumventing a lot of the decades of SEO that has now built up. But, I fear it will be short-lived until people figure out LLM optimisation.
I'm very grateful that we have a lot of players training LLM's, including several that are published as open models and open weights.
I fully expect LLM results to start including ads, but because of the competition I hope/believe the incentives are much better than they are for, say Google's search monopoly.
It could potentially be more insidious though.
We'll probably start sending prompts to multiple models and comparing the results with lower-power local models.
I really hope that they don't include ads in paid tiers. But I'm not sure how much you would actually have to pay to cover the potential lost ad revenue... it might be too much.
This is already the case, SEO content, sponsored comparison sites, influencer marketing, it's all about subtle framing. LLMs just supercharge the problem by making it easier and cheaper to scale.
The real issue isn't that LLMs lie, it's that they emphasize certain truths over others, shaping perception without saying anything factually incorrect. That makes them harder to detect than traditional ads or SEO spam.
Open-source LLMs and transparency in prompt+context will help a bit, but long-term, we probably need something like reputation scores for LLM output, tied to models, data sources, or even the prompt authors.
This already happens unintentionally, e.g. Wikipedia loops, where bad info on Wikipedia gets repeated elsewhere, and then the Wikipedia article gets updated to cite that source.
When LLM-generated content is pervasive everywhere, and the training data for LLMs is coming from the prior output of LLMs, we're going to be in for some fun. Validation and curation of information are soon going to be more important than they've ever been.
But I don't think there'll be too much intentional manipulation of LLMs, given how decentralized LLMs already are. It's going to be difficult enough getting consistency with valid info -- manipulating the entire ecosystem with deliberately contrived info is going to be very challenging.
I use it the same way. The feeling is that I'm back in ~2010 when Googling stuff felt like a superpower. I could find anything back then.
Of course, it didn't last long, and trying to Google now is an exercise in pain and frustration. Lots of people have complained about the various things Google and marketers have done to get there, idk, I just don't like how it works now.
Top LLMs feel amazingly good at rapidly surfacing info online, and as I go through the references they're usually pretty good. I guess the same forces as before will apply, and there will be some window of opportunity before it all goes away again.
I wonder when LMMs and services like chatgpt become as bloated as search engines are today, with their own equivalent of SEO/SEM tools and other unwanted stuff distracting and disturbing accuracy, even if one finally stops hallucinating.
Hopefully not that fast, but I'm pessimistic. The cost of the human bloat will far surpass the current cost of hallucinations. And like we saw with Google, that bloat can become a feature of the content itself, not just contained in the tool.
If you do a web search and find a random blog post full of spelling errors and surrounded by ads, you're not going to trust that at the same level as a Stack Overflow post with a hundred upvotes, or an article with a long comment thread on HN.
But an LLM digests everything, and then spits out information with the same level of detail, same terminology, and same presentation regardless of where it came from. It strips away a lot of the contextual metadata we use to weigh credibility and trust.
Sure, you can follow references from an LLM, but at that point you're just using it a fuzzier form of web search.
> Sure, you can follow references from an LLM, but at that point you're just using it a fuzzier form of web search.
That's exactly the superpower, isn't it? Web search without having to dig deep around (especially now that Google's quality has declined over the years). Sure thing, LLMs are capable of more.
I agree and am perfectly happy using it as fuzzier web search, because it works really well for me.
Finding references is often my main goal, but other times I just want some quick facts, in which case I'll be as thorough with checking as I would when reading a random blog with spelling errors.
Actually, someone should design one that can pull in quotes. Like as a separate tool the LLM uses to quote that has guarantees, it's just a copy and can't be hallucinated. Then you could see the primary info when it's needed/asked for, similar to articles
There will be a race between the attempts monetize online LLM services like this and the development of consumer owned hardware that can enable local LLMs with sufficient power to deliver the same service but ad free.
Combined with RAG a self hosted LLM will definitely be able to deliver a more impartial and therefore better solution.
I don't think anyone can make a Google today that works as well as it did back then. Google shaped how new content was created, and that was probably a much bigger deal than any changes to the tool itself
Sure, but in the context of this thread where the usage case of modern LLMs was described as:
"Now with LLMs, I simply prompt the same with a little bit more of context "pros and cons of using mysql vs mongodb when storing photos. Link references"
locally hosted LLMs with RAG will absolutely be able to do this, better than Googling even back then could, and so the prospect of monetized LLMs with ads in them degrading the user experience for this sort of usage case is unlikely.
I do this. But the killer usecase for me is writing all boilerplate and implementing some half-working stuff keeps my attention on the issue which makes me able to complete more complex things.
A recent example is when I implemented a (Kubernetes) CSI driver that makes /nix available in a container so you can run an empty image and skip a lot of infra to manage.
I talked to it a bit and eventually it wrote a Nix derivation that runs the CSI codegen for Python and packages it so I could import it. Then I asked it to implement the gRPC interface it had generated and managed to get a "Hello World' when mounting this volume (just an empty dir). I also asked it to generate the YAML for the StorageClass, CSIDriver, Deployment and DaemonSet.
So LLM left me with a CSI driver that does nothing in Python (rather than Go which is what everything Kubernetes is implemented in) that I could then rewrite to run a Nix build and copy storepaths into a folder that's mounted into the container.
Sure implementing a gRPC interface might not be the hardest thing in hindsight, but I've never done it before and it's now a fully functional(ish) implementation of what i described.
It even managed to switch gRPC implementations because the Python one was funky with protoc versions in Nix(Python bundles the grpc codegen it's so stupid) so i asked it to do the same thing for grpclib instead which worked.
The problem with Lisp (or at least Clojure) is that abstracting away the boilerplate requires you to correctly identify the boilerplate.
It’s nontrivial to structure your entire AST so that the parts you abstract away are the parts you’re not going to need direct access to three months later. And I never really figured out, or saw anyone else figure out, how to do that in a way which establishes a clear pattern for the rest of your team to follow.
Especially when it comes to that last part, I’ve found pragmatic OOP with functional elements, like Ruby, or task-specific FP, like Elm, to be more useful than Clojure at work or various Lisps for hobby projects. Because patterns for identifying boilerplate are built in. Personal opinion, of course.
Yes, good tooling shouldn't have boilerplate. Minimizing loc (within reason, not code golf) is the best thing you can do for maintainability. Unfortunately things like Java are popular too.
I hear you. But removing boilerplate via abstraction (Lisp) is very different from generating it on demand (LLMs). The former is obviously qualitatively better. But it requires up front design, implementation testing etc. The latter is qualitatively insufficient, but it gets you there with very little effort plus some manual fixes.
> The latter is qualitatively insufficient, but it gets you there with very little effort plus some manual fixes.
I remember years ago, when I worked at a large PC OEM, I had a conversation with one of our quality managers -- if an updated business process consumes half the resources, but fails twice as often, have you improved your efficiency, or just broken even?
"Qualitatively insufficient, but gets you there" sounds like a contradiction in terms, assuming "there" is a well-defined end state you're trying to achieve within equally well-defined control limits.
There’s necessary complexity like error handling, authz, some observability things, etc. which can’t be trivially abstracted away and needs to be present and adjusted for each capability/feature.
i’ve stopped writing “real” code for the most part, i just bang out some pseudo code like:
read all files in directory ending in .tmpl
render these as go templates
if any with kind: deployment
add annotation blah: bar
publish to local kubeapi using sa account foo
and tell it to translate it to x lang.
so i control the logic, it handles the syntax.
asking it to solve problems for you never seems to really work, but it remembers syntax and if i need some kinda reader interface over another or whatever.
can’t help me with code reviews tho, so i spent most of my time reading code instead of remembering syntax. i’m ok with it.
yeah, that’s what works for me also. LLMs are a nightmare for debugging but a breeze for this.
another good use case: have it read a ton of code and summarize it. if you’re dealing with a new (to you) area in a legacy application, and trying to fix a problem in how it interacts with a complex open-source library, have the LLM read them both and summarize how they work. (while fact-checking it along the way.)
> Am I the only one using LLMs as if they were a search engine?
Nope, you're not alone as I also do this. I'm also not using any AI IDE's (yet).
It's funny, I actually recently failed a live technical interview where I was using my LLM of choice to answer Google-like queries instead of an IDE like cursor. The interviewer told me that he had actually never seen anyone use AI for coding like that before. Up to that point, I assumed that most coders were mainly using AI as a search engine and not necessarily using AI IDE's yet. Are we really that rare?
Not at all, I've been doing this with ChatGPT and Claude for a long time. I only recently (last couple weeks) started playing around with Claude Code on command line (not in an IDE). I didn't like Cursor very much. YMMV
In the past you looking around documentation, SO answers etc would have hopefully helped you learn more about the tools and develop skills required to independently analyze the pros and cons. If you ask an LLM (or a search engine or a colleague) and take their words as the ground truth then you won’t develop the skills. Worst, sooner or later no one will have enough knowledge nor analytic skill to form an opinion on any sufficiently deep subject and they will all be dependent on corporate chatbots to spoon feed them information that may or may not be biased in a way against your interest. Now imagine if ChatGPT tells you to use azure and Gemini tells you to use GCP…
This, and I use them for code review and rapidly generating prototypes that I heavily edit. Almost none of the LLM code survives usually. You could ask "Why dont you just write it yourself then?" but sometimes getting started with the skeleton of a working project is the most difficult part of it.
It is nice when it works, but sometimes I run into trouble where I don't know a right word to put in the prompt to get the answer I'm looking for. I've recently been playing around with Raku and had a really cryptic type signature error and
Claude was of absolutely no help because it didn't know about the interaction of Signature literals and 'Slurpy' sigils in method parameters. Only when I learned about it and included the word Slurpy in my prompt would it actually regurgitate the information I was looking for but at that point I already knew it.
I think the key difference here is that if you type into Google the wrong thing it will return poor results that make it fairly clear that you're not on the right track
LLMs will sometimes just invent something that basically gaslights you into thinking you're on the right track
This plus easier search of poorly/not-at-all documented APIs is like 80% of my usage too. Besides that, a lot of “here’s my design for xyz system, am I a stupid idiot for using this architecture?”.
Yup this is where 90% of the productivity benefits come from for me. Instead of needing to spend an hour scouring documentation I can ask an LLM and have an answer in 5 minutes
> I would read the official documentation of each db, forums, blog posts, stackoverflow entries, etc. It was time consuming on the searching side. The time it took to read all the sources was fine for me (it's learning time, so that's always welcomed).
This learning time that you welcomed is what you will now miss out on. The LLM gives you an answer, you don't know how good it is, you use it, and soon enough, if running into a similar issue, you will need to ask the LLM again since you were missing out on all that learning the first time which would have enabled you to internalize all the concepts.
It's like the regex engine example from the article. An LLM can create such a thing for you. You can read through it, it might even work, but the learning from this is orders of magnitudes less than what you get if you build this yourself.
I think it depends. LLMs can link references of where they took the content they throw at you. You can go and read such references. I like what LLMs provide, and at the same time I don't wanna blindly follow them, so I always allocate time for learning whether it's with LLMs or not.
You'd be surprised. A number of fairly technical people who are just not that familiar with ML I know got confused by this and believed the models were actually being tuned daily. I don't think that's universally understood at all.
That has actual practical implications and isn't just pedantry. People might like some model and avoid better dialog engines like perplexity believing they'd have to switch.
I think you either didn't read my response or missed the point. No matter if the LLM output is useful or not, the learning outcome is hugely impacted. Negatively.
It's like copying on your homework assignments. Looks like it gets the job done, but the point of a homework assignment is not the result you deliver, it's the process of creating that result which makes you learn something.
Best metaphor I have found to how I use them is as "a hunting dog".
They can get into small crevasses and the foliage and whatnot and they don't mind getting wet. They can fluster rabbits out. And are somewhat smart. But you still have to make the kill and you have to lead the dog, not the other way around.
"Copilot" is a great marketing name, but a bit deceiving.
For really basic overviews of different technologies I've found YouTube videos to be useful.
For code examples I've always found GitHub code search to be super useful.
I haven't really found a use case for LLMs where it's any faster than the research tools already available on the internet.
A good example is writing AWS CDK code, which is a pain in the ass. But there are so many examples to pull from on GitHub code search that are being used in real projects that I've found this method to be faster than prompting an LLM that may or may not be correct.
I just used an LLM to sleuth out a concurrency issue earlier - I could see what was happening in the debugger, I just couldn’t really see why - asked the LLM to talk me through lifecycle stuff, and boom, first thing it brought up was apparently the answer. Thank you, glorified fuzzy search! Much quicker than pouring through the docs.
I got lucky sure, and sometimes I don’t get so lucky - but it works often enough to have become a part of my regular approach at this point. Why is the thing broke? Maybe the robot knows!
Even when using them to code, I use them as a search engine. Rather than telling them to implement feature X, I clone a repo which has a similar feature and say:
"explore the repo I've cloned at /src/foo and explain how it achieves barFeature. Now look a the project at /src/baz and tell me why it would be difficult to use foo's approach in baz"
I rarely have it do anything novel, just translate ideas from existing projects into mine. Novel work is for me to enjoy coding directly.
I learned about window functions in SQL using an LLM. I hadn't written SQL in over a decade and never ran across them. It explained how they work and the trade-offs. It was great!
My main use for LLMs is to ask it to give me some boilerplate configuration for some library/tool I am not very familiar with and then look up the docs for the options it spits out. Like:
"give me the terraform configuration for an AWS ECS cluster with 2 services that can talk to each other, with one of them available publicly and the other one private"
Occasionally to give some small self-contained algorithms, for example:
Give me a way to format the difference between dates as human-readable text. For example: 1 day, 16 hours.
> Am I the only one using LLMs as if they were a search engine?
Google agree with you and now at the top there is a AI generated answer, clearly labeled as AI generated, and it cites the sources. I was trying to escape AI, but I have to recognize the implementation by Google is quite good.
> Am I the only one using LLMs as if they were a search engine?
I quite like the search-first LLMs like Gemini and Copilot for this reason. They give you links which you can use to verify the output and seem to be less prone to directing you to SEO spam than Google Search/Bing.
Although I do partly think that search engines today are delivering poor performance compared to what they delivered historically so LLMs are benefiting from that.
I primarily use it this way or as a rubber duck. I pretty much use it as a replacement for going on a Google adventure. When I use it for code, it's mostly as a glorified auto-complete or write some boilerplate I can't be bothered to type.
It's kind of scary how good it is. I haven't completely switched over, but I think a lot of that is just not wanting to admit that this is the new paradigm and the implications. The ability to not only find what you're looking for faster, but have it tailored to your specific context? It's hard to go back from that.
i'm currently using it in large part as a question asking tool. feed it a bunch of documents, then ask it to ask me questions to figure out whats been missed -- stuff that i probably know, or could find out, or look for more documents that have the answer, but arent clear from the documents i had handy.
this afternoon i spent some time feeding context, with the idea of making a QnA context for maybe 50 different website routes, so i can have it ask me similar questions per route, so i can have a hell of a lot of pretty accurate documentation for an intern to pick up a couple to work on, or get some LLM to take a try at filling out the details.
im feeding it knowledge, with the output being documents, rather than me getting knowledge from it.
My most common use is to treat it like one of those recipe clippers that grabs a recipe out of an overly long blog post. Last weekend I punched '28 years after credits?' into Kagi and got back only what I wanted. Finally I can have a recipe clipper for any topic. I'm happy with it.
That's basically how I use it though I cannot wait for Gemini to be a click away in my browser with the current screen/page(s)/tab(s) embedded so I can ask it stuff about the current long article/documentation page. We're becoming very, very lazy.
I tell to myself to use them as "teachers", not "interns", i.e. ask them questions to guide my process or look for the sources of knowledge needed to understand or do something, instead of asking them to get things done (except tedious, simple tasks).
Make sure to spend some time asking them questions on topics you already know a lot about. (I like to ask the AI about a game I developed called Neptune's Pride.)
A year ago the AI would just make up some completely random stuff. The current crop do a very good job, but still not 100% correct.
They are wrong enough that I would be wary of using them to "teach" me topics I don't know about.
Nope, I do this too (most of the time.) I don’t like working on code I don’t understand. I have started to ask it to use a client API I’ve written to figure out how clients would work with the stuff I write though. It’s great.
I think one of the major issues I (and others) have with LLM's as search is they are heavily biased towards outdated data. The other day I was trying to have it get me the most updated versions of X, X, X from an old project. I tried several providers and all of them without exception gave me an "upgrade" to an older version than was in the project for at least 1 item(by years). I'm sure they are choosing LTS releases over the new ones because that is most popular overall in a dataset. However no matter how "hard" I prompt they keep giving me those old ones over "latest"
My LLM use case preference is exactly that - a google replacement. I've modified my prompts to ask for links to direct source material, thereby allowing me to go deeper. I find this in no way different then a search engine.
The downside however, is that at least 50% of the links cited no longer exists. This points to a general issue with LLMs, temporal knowledge. I tend to at least go to archive.org on a per-link basis when so inclined.. But clearly that's not ideal.
Does anyone have a better solution, short of just asking the LLM to remove the extra step, and link to the archive.org cache for the ingestion date (which frankly, I also don't have faith in being accurate).
It's a suggestion engine. An autocomplete (I am not saying GPT is just autocomplete of course it is more advanced) but end if the day you need to verify everything it does right now. May not be true in the future though.
Search finds you sources then you can decide if you trust them. AI generates code and you have no idea if what it generated was perfect, slop or almost perfect but bad in some major way.
Nope! I use llms like I used to use Google and StackOverflow. Really great to bounce ideas off of, brainstorm, yes save time too ... But the at the end of the day, what will separate programmers in the future is those that bother to put an ounce of effort into things, read, and learn. Versus those who want the easy button and answer given to them. They're going to be the next generation of people who leak your passwords, social security numbers, PII, and create massive global security incidents.
To be clear. AI doesn't kill people. People kill people. In this case, lazy people. Welcome to Idiocracy my friends.
One of the best things that I’ve done for myself, career-wise, came during a 6-month sabbatical between jobs. I had a bunch of projects that I wanted to do, but I kept finding myself stymied by scope creep - when you don’t have any constraints, it’s hard to decide how big a project should be, and the projects would grow to these massive unfinishable beasts.
I decided to constrain myself to one week per project - whatever I could get done in a week was what I’d do for this project. The experience of going from zero to something useable in either a new language or a new framework or a new field in a week was enormously confidence-building - learning that I was actually pretty good at this whole programming thing and breaking through a bunch of walls I’d run into previously in my professional career gave me an incredible amount of trust in myself to be able to take on new challenges that really helped me when I was looking for a new job. At the same time, it reminded me what I loved about programming - in a week, I was able to build something that solved a problem I had or that someone I knew had, and I got to do so in a creative and intellectually challenging fashion.
If ever you find yourself in a position to take a couple months off between jobs, skip the leetcoding or whatever else and just build some toy projects - you’ll be amazed how much you already know.
I would just add that gen AI can actually be a fantastic aid when you're building some toy projects just for yourself.
I am primarily a backend engineer, though I am capable on the front end, just not particularly fast. I am especially slow at the voodoo that is CSS. In the past, I would have been turned off from doing some end-to-end personal projects because I know I would have been bogged down just battling CSS for hours and hours (it's particularly problematic that I'm slow at CSS but also a stickler for how I want things to look). With AI tools I can literally just say "make it pretty". It gets me about 85% of the way there, but importantly it gets me to the point where it puts in all the styling framework and then I just have to fix bugs/edit/tweak it, which goes very quickly.
So now I've found myself much more likely to build personal projects because I know I can get past things that in the past felt a bit like quicksand to me.
On the one hand, the machines can spin up these bridges over unfamiliar code chasms.
But on the other hand, you (at least I) have no investment in the code. No familiarity with it. No "skin in the game" when I can tell the AI to just redo it.
One way around that is by using the "Chat bots as search engine", and then, rather than cut and pasting stuff into your project, type it in by hand. Even if you're just straight up copying it, by typing it in you have a bit more connection to it. When I do that I can better see the unfamiliar patterns rise out of the code. It's much better for me than just reading it.
Also gives me opportunity to tweak it as I like, which can break things, which can teach me things.
Mind, I've been advocating this "type it, don't paste it" philosophy for years and years, for any code snippet found in a book or tutorial or wherever. It's just now with the bots, the code is more tuned to our exact use case, and the incentive for moving it over is less.
But I find grabbing a blob of code, pasting it into my project, and getting errors so that I can tell the bot whats wrong is not particularly satisfying.
Watching Claude figure out its own errors even less so. (It's very cool to watch, just not very joyful to me outside of "wow, that's cool".)
On the other hand, I have no problem pasting a snippet back with a "what exactly is this doing here" to get better explanations of what I'm seeing.
> rather than cut and pasting stuff into your project, type it in by hand
This type of mechanical copying is probably very important for learning and staying familiar with what it is doing.
Just last night I had ChatGPT help me with some algebra and geometry for a hobby project I'm working on. I'm rusty and got stumped so I asked the bot to unblock me. It helpfully walked through the entire simplification / substitution / solving process step-by-step, which I manually copied down using pencil and paper. Following along like that, mechanically, really helped it click for me. I plan to do more of that.
I've come to the point of using current AI tools as "search on steroids". For coding tasks, I now ask questions using more generic language, looking for references, and when the AI comes back with specific examples, I try to refine the question in terms of the example that the AI has given me. "Kinda like that, but inside of Book, I actually want to do X with the Author"
This makes it so that I have to manually translate code that the AI is showing me to my specific case, typing it out myself in my code's context. This has helped immensely with developing my familiarity with a couple of new (to me) frameworks, and with slowly adding to my overall understanding of what I'm /actually/ building.
The joy of personal projects is they can look like shit! You'd also be surprised how much you can learn by just screwing around with something for a day. A big part of the whole exercise for me was getting out of my comfort zone and learning that I can learn. I'm primarily backend as well, and came up through the ops side (as opposed to formal CS), which always led to a lot of imposter syndrome, so a lot of the benefit came from just putting myself in an uncomfortable position and just staying there long enough to work my way through it.
That said, my goal for this was to wear the hair shirt (and also this was before GenAI was really a Thing) - if you just want to make something neat, absolutely vibe your way there.
When I have gen AI create a starting point for the CSS for me, I still go over every line, and I always have to do a bunch of fixes and edits to get this to look how I want. Frankly, dicking around with the esotera of CSS is something I absolutely don't want to do and don't care about - it's really just reminding me of syntax that I forget because I don't do it often enough. This is also why I find gen AI invaluable for shell scripting. Hunting through reams of man pages doesn't make me learn any faster than just looking at the specific arg flags gen AI outputs (and potentially asking AI more about the details of that command). I just don't do enough shell scripting to really remember the command/arg specifics long term for things I don't commonly do.
> So now I've found myself much more likely to build personal projects because I know I can get past things that in the past felt a bit like quicksand to me.
I'm the opposite of you (my strength is in voodoo) but my experience is the same. Whereas before I'd spend 2-3 weekends getting bogged down on elements that aren't my strengths and never complete, now I can easily whip up a mostly functioning web app (just for me) in just a few hours.
This is how 100% of non-technical executives view software engineers, really. And probably explains a lot of the hysteria (unfounded as I may believe it to be) around LLMs eating the industry.
A good reality check is: if a stranger asks you about a specific part of your toy project, would you be able to explain it?
If you can't, it means there's something there you don't understand, and you lost an opportunity to learn. Of course, this is also true for reusing libraries and stuff.
Within a job, what matters is the _team learning_, and it's roughly the same idea. At least one person in the team should be able to explain anything the team does. Trusting it to an AI puts the whole team in a precarious situation, even if it is something as simple as a CSS layout.
I would argue that, at least for me, gen AI helps me learn things much faster.
E.g., like I said, previously I would have been turned off from doing some projects because I wouldn't have spent a ton of time dicking around with things that are just inefficient. Now I have gen AI create a starting point, and it's very clear to me "Ah, this is how I would use grid layout to get what I want". I can tweak things and it's very clear to me how things are working.
>A good reality check is: if a stranger asks you about a specific part of your toy project, would you be able to explain it?
I like that as a benchmark. I think it also works with LLMs too. I have had best results with AI generation of code when I pass that check. When using a LLM, the chatbot is the stranger.
In fact much of the frustration I have with using AIs seems to be for models tuned to the situation where the user does not have the ability to explain it specifically and it has to make broad assumptions. This may be the best result for a lowest common denominator, but it's frustrating when it provides a generic solution to a specific problem.
I can see specialised models diverging to focus on different levels of assumed knowledge and expectations. Perhaps to have modes of thinking, maybe as a more tone based MOE. Is anyone doing mixture of finetunes?
unless you own/write the whole software stack(not the web stack), i doubt anyone can fully explain their project.
For example, if in your project you have to write a file, and someone ask you how does the kernel cache file writting, that would not be an easy question.
You're missing the point and taking it to an extreme.
Let's say I write a toy project to log all sites I visit to a text file. The goal is to learn browser extensions, not learn kernel cache. There's a clear boundary, and anyone asking me about kernel cache would sound very weird. It's obviously a lot of layers of knowledge deep into the stuff I'm learning.
If my toy project is a beautiful web app, then CSS is _not_ several layers deep. It is in the front of it. It's a dependency I _need to learn_ in order to own my project.
If my toy project is a web app, and doesn't matter if it looks beautiful or not, then I want to spend my time away from CSS (either writing it by hand or interacting with an AI). I am fine with having it ugly.
Understanding these dependencies and layers of knowledge is part of the learning as well.
Again, this works for teams as well. There are certain things, at a certain level, that if you don't have someone in the team that knows it, leads to a precarious situation.
The fact that people don't recognize this, and mistake it for not made here syndrome and other nonsense, is actually scary. It means those recipes (don't reinvent the wheel, good is better than perfect, etc) became mantras that people repeat without actually understand why they exist, and those boundaries have become arbitrary and meaningless.
my current opinion is that uf i can get an ai to write details into a file that another team member's ai can read and then know whats up, thats close to the same as the team member knowing it, as long as theyre either reading the file that the ai read, or working through the ai and providing that file(s) as context.
> If you can't, it means there's something there you don't understand, and you lost an opportunity to learn. Of course, this is also true for reusing libraries and stuff.
Yeah, in libraries that's the aim, software composability and reusability, you don't need to keep everything in your head. Same here.
A "lost opportunity to learn" can be preferable to not making the thing or making it much slower.
I ended up migrating to fixing things in the libraries I use because they piss me off.
Bad onboarding docs, broken or bent SDLC, major performance issues. I spent all day yesterday fixing perf issues in a library that competes with one I loathe for trying to complect an entire problem domain.
You probably have the same thing I do. Teamwork is a very different motivation than working on something either never intended for others or with a long MVP cycle that makes it too easy to get bogged down with side quests. "It would be way less tedious to deal with X if I did Y first. But now Y is blocked so I do nothing."
In my younger years, classic ASP + SQL was easy, and I had IIS + SQL Server set up on a server at my house. And I could write HTML/CSS/JavaScript.
Deployment was... run update scripts, push ASP files via FTP.
Now I know there are newer ways to do things, and I'd love to set up CI/CD in my personal life, but every time I want to do a "toy project" I get hung up researching and thinking about my own personal software development life cycle.
What you choose for hosting and deployment for your toy projects?
I've got a VPS and domain I've had for a long time - for most of these, docker compose + the VPS did the job just fine. I was in the same boat - I'd fallen behind on a lot of the ops stack, and project setup wound up being a tar pit for a couple projects before I got frustrated and just said "Fuck it" and did it the dumb way, which wound up being the correct approach.
I found the trick with the personal projects was to pick the thing I was trying to learn with that project and focus on that - if I was trying to learn React, don't also try to learn kubernetes or lambdas or anything else. You really have to be comfortable with your project kind of sucking for all this to work.
So I'd say if you want to learn proper modern deploy + CI/CD + etc, take a project where you already know the rest of the stack (hell, make it Hello World if you have to) and use that. Keep things simple and in scope, get something stupid working, and then iterate to make it better.
For most of my toy projects, it's a systemd service file and rsync + restart command for deploys. Might have to SSH in and run an install command occasionally.
You can get a VPS in any cloud provider and treat it as your server, just copy files to it. It depends on the scope of your projects, though, if you need a lot of resources it can get pretty pricey for just toy projects.
I pay $12 for a server on Digital Ocean, and I host multiple servers behind an Nginx. But I am primarily a frontend/Desktop developer, so they are simple.
I am in such a phase as I write this. What should have been a week-long project has already gone through three and not finished. This is a simple multi-platform app I am building in Flutter.
One challenge I face is to be able to build something useful within a week. In two projects, I could make wirhin two days each.
For the Flutter one mentioned above, it took me two-three days just to get the tooling/SDKs to work and get me to a hello-world program working on Windows. Then it took about a week to program with all features I needed.
And then, damned again -- Took me a couple days to get it to build APK (random errors; the issue in the build tooling is still open). And then a week of craziness to sort out Android permissions. I still have the latter unresolved.
It took me 8 months of active shopping to find a job w/ 25 years of experience. Unless you are really good at interviewing with an interesting resume, I'd suggest caution until the job market recovers.
What do you think the bottleneck was in your search?
Was it finding openings, getting interviews, or passing those interviews?
In my niche I’m finding that there are very few opening in the city I live in, but that getting interviews might not be too hard. Not sure how easy it would be to pass those interviews.
Most of my issues were around interviewing, I got plenty of them but somewhere in the 3+ rounds of interviewing I'd choke. I have always been nervous during interviews and had previously got most of my positions through my network, but none of their companies were hiring. Next time this happens I'm going to try an interview tutor to see if that helps and probably take some Xanax or something to help settle my nerves.
Take it or leave it - I tell myself i already failed the incomming interview and whenever I notice i get nervous I tell myself im just talking with friends
I'm single, rent controlled, and was working at a FAANG-ish company.
The fear was real, for sure, but it honestly made a huge, huge difference for both my mental health and my employability afterwards. I also got to spend some time figuring out what I actually wanted to be doing, so I had a much clearer picture of where my skills and interests aligned after, which helped the search as well.
For what it's worth, the two tricks I found for leet-code interviews were 1) do the simple stupid thing first - get something _working_, and then make it good and 2) explain what you're doing and what you're thinking about to your interviewers. I'm still not Great at the leet-code style, but you can get a lot of the way there with some basic knowledge about algorithmic design and complexity and data structures. In general, I've found finishing the problem and being able to talk coherently about it trumps finding the ideal solution.
(I also code like a psychopath - I'm constantly rewriting what I'm writing while I'm writing it, so a) slowing down and b) telling interviewers what I'm doing and why made that look less chaotic.)
I'm also not the person you want to hire if you're looking for deep algorithmic knowledge, though - I've got a long career of delivering high impact systems and solutions, but if you want someone to rewrite the index parsing algorithm in your mature database software to minimize lookup speed, you've got the wrong guy.
I actually just started telling interviewers that I'm happy to discuss code, problem solving, etc at a high level during an interview or walk them through problems I've solved in the past, but I won't do leetcode problems
It hasn't outright disqualified me yet. I also think it's a good "is this team a good fit for me" filter
Toy Software is a lot like working on your bike/car/boat/whatever.
Working on your bike is fun. Working on the bike you need to ride to work tomorrow is stressful.
I miss writing toy software. It’s fun. But inevitably I want to use said software and that’s where the trouble starts. You find all the bugs and none of the time to fix them.
I recently wrote my own invoicing application. I got wrapped up in the joy of adding the features I wanted. Many of these features come at a premium monthly fee for a big named product.
I needed to get an invoice out in a timely fashion. As much as I wanted to use my app, I found certain kinks I needed to work out (with styling, adding addresses, etc.) -- This was where I realized what you have articulated.
At some point, it becomes better to prioritize the "fun" in working on my bike, and the "usefulness" of the daily driver bike.
In doing so, perhaps the fun/usefulness of each converges over time.
I used to self-host my email. I don't any more, for exactly this reason. And of course this applies to every other "just do it yourself!" enthusiast appeal. I actually want to delegate it to someone who does it full time.
Strange example, because self-hosted email has been pretty much setup-and-forget for me. I’ve been hosting my own email for several domains on a $5 VPS for over a decade and it’s largely just hummed along, without needing babysitting. Very rarely, some ISP’s spamblocker gets a little too squirrelly and I have a brief deliverability issue, but those are rare and easy to resolve.
It was Exim on Debian; generally it would chug along fine for months and then incur some sort of regular sysadmin issue that nonetheless needs dealing with. Upgrades. Occasionally, breaking upgrades. Changes to anti-spam rules on other systems causing deliverability issues. ISP mandating IP migration. Earlier in the process it was an actual, physical rackmount server not a VPS, which incurred extra effort a couple of times to replace it.
And the big one: inbound anti-spam.
It's not that issues are frequent, it's just that there's no guarantee they won't be badly timed. There may come a point in your life where (a) you don't have a lot of free time and (b) you badly need a specific email.
I still have the mail domain, I just pay an ISP to handle it for me.
I heard a lot of people saying hosting your own email server was the best way to be considered as spam by every big email provider. I think it's a big issue, but I've never actually tried it.
I also have more "working software" interests than i have time and energy. Many of them have a ton of mundane boring details i've done a thousand times before. I'm actually a bit excited that i can work on what i want to work on.
With that said it's a challenge and a chore on its own to curate what "AI" writes. So it's definitely not all roses.
Disclaimer: I'm still early in the experimentation phase. We'll see if i hold these opinions in a few months.
I actually only like working on software that has a real use, at least to myself. The coolest is when I spin something up fast at work and people start relying on it.
Same goes for my car, it's satisfying doing inexpensive fixes to keep it running. Wouldn't want some mid-2000s BMW project that's stuck in the garage half the time.
I dunno. Working on a bike you want to use to ride to work could be stressful if it is your only bike, you need it to get to work, and your manager will be really mad if you don’t show up to work one day. If there was any slip available it wouldn’t be so bad.
Actually, I really like this analogy because
1) We can’t really work on non-trivial fixes for our cars because they are too complex
2) We’ve totally structured our living situations around these over-complex beasts
3) Sometimes they do just break down at random, and it becomes a mini-crisis for everyone involved, which can’t really be resolved without missing a day of work to go to a professional
4) The bike is in exactly the same situation, except you can actually fix most bike problems yourself with a little kit
And yet, it is seen as a serious adult thing to have a car, instead of a bicycle.
Some thoughts:
Outside of, like, customer service, most places are throughput-oriented and it doesn’t even really matter all that much if somebody misses a day.
If we really cared about “uptime” of employees, we’d all be living in bicycle range of our jobs, and able to fix our bikes. Maybe there’d be a bus route as well.
As someone who's stuck to owning a single bike for over a decade (tho I will probably change this in the coming year) for both athletic rides and commuting, I'll say the "it is seen as a serious adult thing to have a car" can sometimes make having a single bike worse.
If my bike flats and I'm out of tubes and I have a busy day planned, I have to take it to a local bike shop but those generally close pretty early. If I miss the window to take it to an LBS then I'm SOL. With a car there's generally a developed enough ecosystem that I can get a roadside assistance type service to put a spare on the car until I can actually get my car into a shop.
This is a good analogy for SaaS vs using a tinkerer tool. A tinkerer tool often can be customized exactly to your needs. I can work on my bike at 10 PM in my garage after dinner as long as I have the parts. But sometimes your tinkerer tool is going to leave you working all night or you'll be SOL. But SaaS is generally a full-time job for other folks. Sure you pay money, but there's fewer situations you're SOL.
(Personally when my bike is inop and I need to head to work I use transit.)
> 1) We can’t really work on non-trivial fixes for our cars because they are too complex
Learning how to fix my own cars is something I’ll always be grateful for. They’re not that complex. Most of it is just bolts and a diagram reference.
Sure, if you crack a piston cylinder you just need a new engine, that isn’t really fixable.
Brakes/rotors/calipers, it is almost criminal what you pay a shop for in lieu of doing it yourself. Spark plugs are super easy. Even things like taking the intake manifold off a direct-inject car to replace the fuel injectors isn’t terribly hard.
What has made cars complex is all the electronics that tie into it. The mechanics of an ICE car are mostly about the same.
> If we really cared about “uptime” of employees, we’d all be living in bicycle range of our jobs
And then reality hits. You're married and your partner's job is on the other side of the city. You just bought and house and now you want to switch jobs but you'd be limited to only jobs in biking distance. eTc....
I bike to work (was 4 miles, now 2), but it's unrealistic to expect people to be in bicycle range of their jobs.
I really just write software for myself only. The time difference (and fun level) between writing software for yourself and writing production quality software is huge.
Feature doesn't work in Safari? Who cares, I don't use Safari anyway. No need to fix.
Perfect example for me is that I use beancount for tracking my finances. It broke one day because I tried upgrading some dependencies from v2 to v3. I tried messing around with the imports and some function names but it didn't help and after wasting 30m I decided to give up on the upgrade.
2 weeks later when I wasn't busy trying to get it to work, I gave it another shot. This time went so much more smoothly and now everything works (fingers crossed since I haven't actually tried out ALL the functionality).
I did this for years to learn computer graphics. I spent so many weekends and evenings building weird things that never made me a single dollar, but all the knowledge I gained eventually led to the job of my dreams. Here are some of the weird things I built:
I quite enjoy the spirit of the article, and I believe joy in programming has become even more important in the AI agent coding age we're leaving.
However, am I the only one finding those time estimates way too short?
I'm not the fastest programmer on Earth, but I'm not the slowest either, and I think most of those projects would take me a lot more time than those estimates, specially if I'd be working only 2-3 hours per day.
I feel most of those projects would take significant time researching and learning about the subjects, before even starting to code.
Example: recently I replaced my Pelican blog by my own hacky static site generator written in Odin, working 2-3h per day, it took me 2 weeks -- and this is a simpler project than many on that list.
Your project is vastly more specified than a toy project, because it has a real customer (you) who will expect it to work after it ships. That expectation is what separates toys from real tools.
I bet you could write a word processor in an hour. It might only add or delete one character at a time (no selections). It might persist only to a hard-coded "output.txt" filename. You might have to kill the process because there's no quit operation. But that would suffice for a toy word processor.
Fair point, my example was indeed "shipped to production" and may not compare a throwaway static generator toy project.
I still think those estimates are off, because I think many of those projects would need significant research and learning time, possibly more than actually coding -- not to mention time spent troubleshooting when something goes wrong.
For me, one of the enjoyable aspects of toy projects is that there is no schedule to meet. Which means that I can take my time. And yes, that means I take much more time than the article implies. As an examine, I've been polishing my PEG-based Turing complete language (which features both AOT-to-C and a fully featured repl) since COVID was a daily aspect of life.
When I use dynamically typed languages, it's not necessarily the lack of types that make me write code quicker, it's the fact that I can call anything from anywhere with a complete disregard for whether I should.
When I'm working in statically typed languages, esp. those with explicit error handling (Go, Rust, Haskell, Odin, Zig etc), I find myself spending a lot of time thinking about edge cases, bugs etc because I am forced to do so and find myself thinking in terms of engineering rather than exploring.
I mean, all of those depend on how much was imported as third party dependency, how much was not addressed at all, and what was actually solved manually.
They're all doable if you focus on the core problems. He's got a github with many of the examples up, it's instructive on how to keep things small.
And I think it's important that (based on the code) he had a solid understanding of the problem space before starting to write code. You don't write that tight when you explore a new space.
I'm surprised by the number of bad takes on LLMs in this thread.
LLMs spoon-feed you with information about how things are implemented. You are not supposed to know how everything works when you start these projects. You're supposed to try your best, inevitably fail, then research the topic and understand where you went wrong, then adjust your approach. If you know how everything works and just follow the tutorial, you won't know what makes other methods fail, and by proxy what makes the one you chose work.
Write a language parser with a regex. Find out that it can't parse recursive statements. You've now learnt that regex can only parse a specific subset of syntaxes. Try to workaround this by pattern-matching the most nested statement first. Find out that it blows up performance. You now know more about time complexity and know what to watch out for when you write a real parser.
Write a non-optimizing compiler from scratch. Find out that you can't make do with unsound optimizations because you can't keep track of what optimizations are applied where. Find out that implementing sound optimizations is hard because you need to track use-def chains. Then you'll understand why SSA is used. Find out that code motion is a mess. Learn about sea of nodes. Merge every optimization pass into one because you're unable to order passes right. Learn how e-graphs solve this.
Write a layout engine. Get stuck on being unable to define what a "width" is. Workaround this with min/max/natural widths, introduce binary search, etc. Learn how this stuff works in practice (this is something I haven't personally done yet).
They say we learn from mistakes. Please don't let the smart (or "smart", depending on how you look at it) machine stop you from making them. It's not a teacher and it doesn't know how to educate.
A lot of people say if you don’t use LLMs then you will fall behind. I’m starting to think that not using them will be a significant advantage in the long run.
I think LLMs improve productivity in the present at a significant cost for the future. It's like cutting an R&D department. You might be able to utilize existing approaches better, but you won't make progress, and I think people are way too overconfident in believing everything important has already been developed.
I guess the counterargument here would be that LLMs could improve research as well by optimizing menial tasks. It's kind of similar to how computing has enabled brute-force proofs in math. But I think the fact that students are still required to prove theorems on paper and that problems with brute-force solutions are still studied analytically should show that tools like computers or LLMs are not at all a replacement for the typical research process.
IMO we are going to see a large class of people who have cognitive deficits brought on by AI tool usage.
I've been wondering lately about how to distinguish between tools that enhance your cognitive ability, and tools that degrade it. Jobs called a computer a "bicycle for the mind," and it seems like LLMs are an easy-chair for the mind. I'm not sure a priori how to distinguish between the two classes of tools though. Maybe there is no other tool like an LLM.
i think theres both. the LLM is an incredible tool you should be able to use well, but its a complement to your other knowledge and tools, not a replacement. if you dont add the LLM to your toolset, youre not going to be building at the same scale as people who are, and if you dont have the backing knowledge, your LLM outputs are gonna be junk because you wont be able to point it in the right direction soon enough in the context window
But your brain was the clipboard. That simple process of transcription was something that you couldn't avoid learning from even if you wanted to. You'd notice the connections between the commands you typed and things that happened when you ran the program even if you weren't trying to.
Things would start to click, and then you'd have those moments of curiosity about how the program might behave differently if you adjusted one particular line of code or changed a parameter, and you'd try it, which would usually provoke the next moment of curiosity.
This was how many of us learned how to write code in the first place. Pasting the output from an LLM into your source tree bypasses that process entirely -- it's not the same thing at all.
One thing I've learned about building toy software projects or even just small personal software (stuff that's actually useful) is to avoid implementing flexible configuration engines. I used to fall into this trap where I thought I should make my software to be configuration-centric so that it would be usable for others in case I decided to make my code FOSS. The problem with that is making a configuration engine is a lot less efficient than just writing the code to do the damn thing. I've almost never shared any code outside of small libraries that were very low in complexity, and I wasted a lot of time writing code for other hypothetical people when I should have just been writing it for myself.
It took me a solid decade to accept these lessons. I cannot build a product that even remotely competes with things like Visual Studio and .NET tooling. How many hours would it take a solo developer to recreate a debugger experience that anyone else could be bothered to look at for more than 5 seconds?
The hardest part of building something configurable is the UI/UX. The moment you step outside of the host language, your tooling support goes to zero. Sure, you can pick other popular languages as the language of configuration (SQL, python, Lua, etc.) and piggyback off those ecosystems. But, you still have to build a way for a "non programmer" to visit all of the patchwork sites of "configuration" in a coherent way. Telling the business people to go edit a folder full of .py and .sql files will probably not be received well. Wrapping all of this with a fancy web interface doesn't make the problem less severe either (it makes it substantially worse).
If you have a way to build/deploy your product in <60 seconds from source, worrying about elaborate configuration and customization architecture seems extremely idiotic to me now. The best path I've found is something approximating a common library and a per-customer executable that consumes it and implements the expected contracts. Each customer's project lives in a subfolder on the same monorepo. The power of the hierarchical filesystem is tragically understated throughout our industry.
This is also a great way to stand out among other candidates in the resume-review stage of a hiring process.
The number one question I have as a hiring manager is always "can this person actually write code". A GitHub profile with 2-3 projects with multiple commits to each one answers that question really well. I won't filter out people who don't have that - lots of great engineers have never published code before - but the people who do have that are much more likely to get picked for the phone screen.
The converse is actually a negative signal for me. If you list a GitHub profile on your resume, but that GitHub profile is a ghost town consisting exclusively of "test repo please ignore" and "freshman-year assignments I was required to fork from professor's repo," that tells me you don't have an above-average understanding of why you add something to your resume.
If you don't have an impressive GitHub profile, that's fine. To your point - the absence of it is not a penalty, and the presence of a good one is a very positive signal. But the presence of one that turns out to be a dud is a negative.
It partly depends on the level I'm hiring for. A junior candidate with a GitHub profile full of obvious tutorial projects still helps me skip putting them through FizzBuzz. A senior candidate with a ghost town could reflect poorly on them.
I wouldn't consider a GitHub full of "tutorial apps" as a red flag because it's very easy to underestimate the work behind a project when looking at it from an outsider's perspective.
This is especially true for code that was voluntarily written in someone's free time. For example, each work day is made of 8 hours 5 days a week, but a "hobby day" is more like 3 hours every other weekend. So I would consider one or two features per year to be a pretty productive pace for a volunteer, free time project. Or, worst case scenario, I would just be neutral about it and ignore those projects.
I've been using vhs ( https://github.com/charmbracelet/vhs ) to script gif creation- really showcases CLI workflows if you're demoing multiple commands.
This is great! Asciinema is well-made and a lot of fun, but it sadly doesn't show the recordings in motion right there in the markdown/README. Looks like vhs neatly solves that problem, appreciate the share.
> Perhaps you’re a user of LLMs. I get it, they’re neat tools. They’re useful for certain kinds of learning. But I might suggest resisting the temptation to use them for projects like this. Knowledge is not supposed to be fed to you on a plate.
I get where this is coming from, and I even agree with it today, but I also want to tag it as "don't cache this opinion too hard". It's interesting to notice when and how our advice for getting help from AI is different from our advice for getting help from other humans. It would be kind of odd, wouldn't it, to put a paragraph at the bottom of a blog post that said "by the way if you have friends who are expert programmers, I don't recommend asking them for help." I think there are two clear reasons that feels odd: 1) expert friends can actually answer your questions and get you personally unstuck, which is huge, and 2) expert friends usually understand why you're doing what you're doing and that they're supposed to help do it yourself instead of just doing it for you.
One thing I bet few people have tried (because I haven't tried it myself) is actually asking an LLM to guide you like an expert friend would, instead of just spitting out code to solve your problem. Maybe they're bad at that, I wouldn't be surprised. But if so, I bet in a year or two they'll be amazing at it. It might be good to build the habit of clarifying what sort of help you need, instead of assuming an LLM will give you the wrong kind of help?
(I'm the post author) Actually, I would recommend that you don't ask an expert friend for help. If you get really stuck then maybe do some light reading about the topic: but the point is to throw lots of things at the wall and puzzle your way through it. Figuring things out from first principles is fun and also provides you with a litany of creative thinking skills to help you tackle other problems. I firmly believe that spending time confused is an essential ingredient in the process.
Obviously, you're welcome to do as you please though, mileage may vary, etc.
Call it whatever you want, it helps me get unstuck when my dependencies aren't getting along, or when I need to figure out what's wrong with my usage of an API I'm not familiar with, or why my build won't startup based on the code and the vague error message, etc.
I don't care if it's a human or a robot telling me, I just want to get unstuck and I've been getting unstuck more since the advent of LLMs than in the rest of my programming career. Digging through bad documentation and Googling for old form posts of similar-but-not-quite-the-same-issues was always the worst part of programming to me, and I'm glad there's a tool that finally solved that problem.
i am NOT saying they are not useful; they are. especially in domains you are unfamiliar with, or boilerplate, or when being used by an expert in the loop. but that is NOT the same thing as being an expert.
an expert can write 10 lines of code that will be hyper fast while an LLM will write 30 lines that are super slow. both "will work", but one will fall over under load and will be hard to maintain and the other will not.
This is by far my most common usage of LLMs, and they’re good at it. Sometimes you have to be intentional about preventing it from being sycophantic and just telling you that you’re right, through a system prompt or by phrasing the question such that it’s comparing ideas neutrally instead of comparing “your” idea to some other idea.
It feels like Claude and ChatGPT have both become more sycophantic over the past months.
This is quite an impressive list, and many of the things ranking low in difficulty for the author would have been quite high for me. It's definitely inspiring, makes me want to dust off a toy of my own.
That said, I feel the conclusion on learning with LLMs deserves some more nuance. It really depends on how you use them.
This is a terrible prompt for learning:
> Implement this solution for me
This is an AMAZING prompt for learning:
> Please give me an overview of ELF. Focus on the highest level abstractions, more of the 'why' than the 'how'
Certainly one can argue that it takes something away, in that you did not have to conduct the research yourself whenever a question occurred. However, if you are engaging in honest intellectual work (i.e. really thinking), then having a Socratic teacher always ready to discuss your questions is a profound accelerant to the learning process.
I think what you've identified is the "certain kinds of learning" I mention in the post.
If I want to learn about the structure of ELF binaries, I don't figure it out from first principles. I can't: the decisions that went into its design where highly specific to its history and the sensibilities of its creators, and in a modern context a lot of those decisions are near enough random. These are the sorts of things you should use specs, docs, Google, and maybe an LLM for.
What I'm talking about is, I guess, 'constructive' learning. I might not be able to build my own ELF parser through guesswork, but I probably can write a binary format of my own and a simple OS kernel to read it, given enough time. It probably won't look much like ELF, but the process of discovery that'll happen will teach me about why ELF came to be and what problems is solving, and also about the wider design space that sits around it.
My suggestion is that you should not use an LLM's help for the latter.
I like to think of it this way: if you bolted me to a chair in a room with nothing but a laptop, a text editor, and a compiler, what might I be able to create if you left me for 10 years? What couldn't I create because I wouldn't have sufficient implementation-specific information?
Vibe coding via Claude has re-ignited my passion for fun, side programming projects. After a few false starts, I figured out the tooling, processes and in just a few weeks I've built several apps and I'm having new ideas consistently:
-Family calendar/weather dashboard
-Bluesky reader that allows you to hide posts you've "seen"
-Work PM dashboard to make time tracking more gamified and dare I say... fun?
-Chrome Reddit extension that hides posts after viewing for X seconds
-A WordPress plugin to replace a plugin no longer maintained
What I love and had to get used to was looking/using these apps and being happy with them being only 90%. Initially I had Claude make many UI enhancements but I learned to just let most stuff go and focus on the function and building new stuff, not refinement.
I struggle with Claude fixing a bug and not giving me the updated output. Like explicitly having to ask it for the fix 6 times while it insists the update is in the output. Did you ever have that happen?
I read some good advice: give Claude 3 bites at the apple and then burn the session and start over. I find if Claude doesn't get it ready the first or second time the chances of a successful outcome drop considerably. Also "Projects" was a nice low setup way of giving context.
^^ this. Also, not only burn the session - sometimes you need to burn Claude. I'll often find the same problem that Claude struggles with is pretty easy for O3 + GPT 4.1 (or vice versa).
I haven't seen this, unless I don't understand exactly what you mean. I can ask it questions about my code and get answers most easily if I explicitly say "don't write any code".
I have a couple of projects like this and I'm not sure if they'll ever see the light of day. I enjoy writing code, simple as that. Only a small part of my day job is coding, but honestly, it's more like a hobby than anything.
The whole thing with getting people to like your work, using it for clout or padding your resume has an appeal to it that I won't dismiss, but all that takes the fun out of it.
I just don't have much else I actually enjoy in life, so ruining it with peer-pressure, chasing clout, impressing people, worrying about career prospects is a buzz-kill. I used to think that if I am passionate about something like this and I get really good at it, maybe it will help my career and prospects but the real world doesn't work that way. it sucks the joy, fun and passion out of you if you let it.
All that to say it is perfectly fine to like coding for the sake of enjoying it and nothing more. Others play a sport (or watch it), work on arts & crafts, or raise chickens or whatever, I code. I am not particularly skilled or great at coding either, I just enjoy it, and that's it.
If you have fun coding, be protective of your joy! even if you don't take it to the extreme like I do. that's my only point.
I love this take, and I’m reminded of the important idea that while we all are encouraged to be productive, our own joy is also an important thing for us to produce.
I've abandoned tons of toy code but I have 1 thing I wrote in a weekend for myself that I've been mostly blindly using ever since. I just realized while typing this that it's been 8 years. It's a Firefox add-on that I only published to the "store" because running a local-only add-on is a PITA.
It takes regex(s), searches attribute values and/or inner text, and applies a given CSS rule to elements that match. The UI is awful (regex escaped to be inside a json string escaped to be inside a textarea all parsed by javascript) but I know how to use it.
It can probably be used for many things, but I just use it to set visibility: none on elements that link to sites I don't ever want to see (facebook, twitter, linkedin, popular pay/register-walled sites, etc). For example, my HN and reddit views just have blank lines where links any of those sites would normally be. For the most part, I add a site once, and forget they ever existed (except when their CEOs do awful things).
Possibly tangential, but I've really been enjoying writing software with very minimal tech stacks with no dependencies (or very few). Go works well for this, but really any language that lets you create a file and just start writing straightforward code is sufficient. As soon as I need to use some CLI to scaffold out a project with tens or hundreds of files, or pull in a ton of dependencies, I kind of check out. It's fine when you have a team and a big serious project, but in my free time I really just enjoy having a couple files of straightforward code and that's it.
Big time! Almost all of my interesting knowledge came from toy projects I made to solve my own problems (or zero problem solved but it was fun so I did it anyway).
For example, I wanted to know if I can make 1 Kubernetes cluster span multi region, multi cloud. So I slapped TailScale for networking, replaced etcd with multi-region PostgreSQL Aurora, and span the Kubelets between my Raspberry Pi, Digital Ocean VMs, and AWS EC2. And then as the "customer app", I run CockroachDB, rqlite, and tiDB on it (one at a time, I don't want to burn money for this).
It was janky, zero SLA, I likely mis-tuned all the databases, and cost a bit of money :(. But it totally worked, all the db nodes can discover each other. I was satisfied and that's enough.
Another example would be writing my own Raft-backed database, similar to the ToyDB Rust project posted here. Is the DB useful? No. But it is so much fun and I learned a lot.
My AWS S3 bucket and private Git repo are a graveyard full of toy projects. My own Dropbox clone, Pinterest clone, Delicious clone, subset of Google Maps clone, etc. etc. are all RIP in there.
rqlite[1] creator here, happy to answer any questions.
If you're interested in writing Raft-backed databases you might be interested in my talk at GopherCon 2023. It walks through doing exactly that, step by step: https://www.youtube.com/watch?v=8XbxQ1Epi5w
As of right now? Nothing important. If I want to use it day-to-day, then I have to care about SLA, then I have to be the sysadmin of that thing.
Not exactly fun :)
What's interesting is that the TailScale network allows the local daemon be exposed on the internet with small settings change. Pretty revolutionary for personal/home IoT projects.
> In 2025, the beauty and craft of writing software is being eroded. AI is threatening to replace us (or, at least, the most joyful aspects of our craft) and software development is being increasingly commodified, measured, packaged, and industrialised. Software development needs more simple joy, and I’ve found that creating toy programs is a great way to remember why I started working with computers again.
Sad but true. I especially feel that comment about losing the most “joyful” part.
> AI is threatening to replace us (or, at least, the most joyful aspects of our craft)
I don’t like this either but every time I use LLMs it feels like we’re talking about completely different things. It moves waaay to fast and makes bad decisions at every turn, if I accepted them all it would be complexity deadlock within a week tops. Pooping out boilerplate sure but then you’re generally holding it wrong anyway (or there’s an opportunity to automate things). Plus even if you don’t have the time to automate it, sure, but then are you enjoying the act of shitting out your own boilerplate?
Out of the things I consider fun the LLM is at best a good rubber duck. It needs constant hand-holding, sometimes polluting the context window (and physical space) with a barrage of poorly written code. Code is bad, we’re trying to minimize it. No? At least that’s how I think: what’s the minimum* amount of code that can solve this problem?
*: minimum in a brain complexity sense, not char count. They correlate strongly though
I find using the LLM as a rubber duck helpful, even if, no, especially if I do not even end up pressing send. Just writing out the problem leads me to the answer.
I have more joy than ever working on toy software while using LLMs. My joy comes from building things, thinking architecture, components, creative solutions that create something new and exciting. Hand-setting lines of code is NOT what brings me joy personally, it's a means to an end. Dancing with agentic LLMs is so much fun to me. I can however understand that if the code part is what brings you joy, then we're probably not moving in a direction that pleases you. For those of us who like to build, first and foremost, it's heaven.
I don't think the hand-setting lines of code is really the part that brings these people joy, the joy is in understanding. LLM-like tools do not create understanding in the user, who is farming out the understanding in order to get results quickly.
I understand the draw and usefulness of this, but just clarifying, it's not the typing that matters, it's what is going on (or not going on) in one's mind.
The first example in the article, a regex engine, clicked for me. I had read the book on beautiful code #, and the first short story was about a regex thing by none other than Dennis Ritchie. As Fate would have it, my job required me to make a tool for web scraping. Inspired by that book I created a pattern matching language for dom trees. And a click and point tool to create those patterns. At the best time or company had a full time person maintaining maybe 10k patterns to scrape days from library catalogs... It was not a toy project, but it did feel like, even if I got paid to do it.
This is one thing I don't get about the LLM people. For most of us who are professionals, writing software is not very hard, and it is in fact fun!
I don't need to let a computer write my software. I already know how, and I enjoy it. I need the computer to do the hard part (gathering requirements, speaking with stakeholders, etc).
There are several sources of joy in programming. Some lean more toward the process and exploration of the problem domain, others more toward the outcome and impact.
I had a shift of perspective at some point where I realized that software wasn't just fun to create, but you can solve problems with it, including your own, which is incredibly rewarding and kinda feels like wizardry.
Ultimately I find both to be enjoyable, though in terms of career and life outcomes, focusing on solving real problems and producing tangible outcomes has had more of an impact than noodling with neat implementations that nobody but me cared about.
Turns out if you solve your own problems (or just publicly work toward solving them), there are often other people who also struggle with the sam problems, who may throw money your way ;-)
I've done a lot of these kinds of projects. I got really sucked into a rabbit hole with the parser/interpreter/compiler that I was stuck (happily) in "I need to understand this so I will build one" mode. I eventually built a database server and query language for it. (for a business) This was in 2009, but I highly identify with his method of learning by doing for fun!
What's sad is how difficult it is to write software today. In the old days your dad could buy a C64 and cobble together an application. It should be vastly easier to do the same kind of thing with vastly better building blocks today. Why can't some Grandma drag and drop some widgets and have a recipe manager with sharing features amongst her friends and family?
1. Because half her friends and family are on iOS, and that means fighting the App Store. (This is a social problem essentially, in fighting Apple)
2. Because networking is hard. How would you have shared recipes with a computer in the C64 days? Email? BBS? (There are partial technical solutions to this, but they would require people to run something like friend-to-friend overlay networks)
3. Because most stuff happens in web browsers and that means pay-to-play, or vendor lock-in, or using AWS free tier and being a programmer. (Ass, grass, or cash, nobody hosts for free. Friend-to-friend networks may also help with this)
4. Because a recipe manager with sharing is best implemented as just emailing your recipes to your friends and storing them as txt files locally. Anything more complicated is beyond the scope of a Visual Basic-style drag-and-drop WYSIWYG anyway
5. When was drag-and-drop enough? The widgets need code behind them to save and open files, right?
6. You might be kinda onto something, and the longer I write async code I more I think the programming world is ready for another big pruning. Like when structured programming said "No goto, no long jumps, if-else and exceptions is all you need", we might be ready for "A single control thread which never blocks and never awaits, and delegates to worker tasks, is all you need until you are building some C100k shit"
When I was a teenager, I imagined that after a decade or two of working with computers, I would be able to write a computer game over a weekend. Or maybe two weekends. I had a notebook full of ideas and sketches, so that when I am ready, I will make all those amazing games.
I even made a few (quite simple, from my today's perspective) games in Pascal during high school and university. I expected to become much more productive over years of practice.
That didn't happen, for several reasons.
First, my expectations for a good game have increased. I don't want to make the most complicated game possible; I am perfectly okay with 2D raster graphics and simple algorithms. But I expect a good game to have animations, sound effects, at least ten levels that feel different, and an option to save game progress. My old games barely had half of that (some were animated, some had ten or more levels, only one had both of that).
Second, things became more complicated. It is no longer "320 x 200 pixels, 256 colors". Windows are resizable; different screens have different sizes. Programs need to be aware that multiple threads exist. Sometimes there are many possible choices, and I get paralyzed by choosing between them. Programs are expect to have installers; it is no longer enough to have one EXE file, and optionally a few data files together in a ZIP file. It felt like every time I mastered something, a new problem appeared to be solved.
Third, as a teenager I didn't realize how much my everyday work would differ from the kind of work necessary to make a computer game. Some skills are transferable: I am more comfortable with using threads, parsing data files, writing algorithms, the program architecture in general. But many skills are not: if my dream is to make a desktop application, then e.g. all the web frameworks that I have learned over those years are useless for this purpose; and they have cost me a lot of time and effort. So from the perspective of making computer games, as an adult I maybe learn in five years as many relevant things as I have learned as a teenager in one year, when I had lots of free time that I could dedicate to this goal.
Fourth, life gets in the way. There is much less free time, and much more things that I need or want to do during that free time.
So here I am, after a few decades of IT jobs, and (a) I can't really make a complete computer game over a weekend, and (b) it's irrelevant, because until my kids grow up I probably won't get a free weekend anyway. Or rather, even the rare "free" weekend (when the kids are away) is spent on other things that have higher priority.
How hard did you look? WordPress has a few recipe maker plugins if you didn't want to code anything. Just install one and password protect the whole thing, and then teach (and write instructions for) Grandma to use it.
In the age of powerful computers, you can use Hypercard on an emulated Mac, you can use any number of hypercard-clones out there. She can just use Google slides. etc.
I think the main difficulty is deployment. Grandma wants that recipe manager to be available to her family 24x7. How can she deploy it easily for free or very low cost? If there were a modern Hypercard, I think the key to its success would be making deployment extremely simple, reliable, and safe.
NextJS with Neon on Vercel has a capable free tier, and there's enough training data that LLMs are decent at it. If Grandma is that interested in building an app, I'm sure she'd love to spend a few hours with a grandkid to set things up and then being taught how to vibe code (and also how to call said grand kid for help).
How about a very reliable device with an interface that even a child can understand, deployment process that consists of only one action, and no upkeep costs? Like, for example, a notebook... Not everything has to be computerized or software.
I was going to make a similar comment. This list is fine, but the times are all over the place. If they removed those estimates it'd be a much better list. If a hash map takes 3-5 days for someone, they aren't making a threaded VM in 7 days.
I have to assume the author listed these times based on how long it would take for someone to follow along with the blogs that they linked to, which demonstrate stripped down simple versions of these things. Like the POSIX shell example just implements a few basic things, but leaves out a lot (and says so).
This is the point, yes! Toy software is not production-worthy. It might not even be sufficient for your own personal use. It's about building proof of concepts and rapidly scouting out the terrain of a field you don't know much about, and getting a souvenir to remember the experience with.
Oftentimes I find that I enjoy one of the projects so much that I take it further than 'toy', but that usually ups the time estimate by an order of magnitude.
I really like the idea purposed here but none of these projects are even remotely interesting to me. It's moments like this when I start to question whether or not I'm losing my love of programming.
Perhaps. I scanned the list and most of them sound like interesting projects that I would like to tackle at some point (or that I have tackled in the past for "fun").
Honestly, I think I was in your same position until recently, then I had some time off work thanks to the newborn and decided to start a "simple" coding project. Since it was purely for fun and to challenge myself, I decided to try and avoid bringing in any dependencies and wrote everything from scratch. This made it much larger and more complex than I had initially anticipated, but despite that (or, maybe, because of that?) I longed for those few hours when I could tinker on my project. Coding suddenly became fun again.
That gave me a good chuckle. Yeah look, you can change your data layout to be in more of an ECS format, make some optimizations and call it a day; but in reality you're still going to be pretty far from the real thing.
I can say this because I've been working on one for some months now, of course I'm going for a production build and also to turn it into a Unity asset that's worthy of being sold, so it's going to require a lot more effort. The goal of mine is to ensure that its extremely simple to use (which Units DOTS totally fails at) while still being as efficient as possible.
Some notable things I've had to do for this project:
- I have a tool releasing on the Unity asset store (currently being vetted) that performs optimization for you, e.g. you give it a variable such as an int, a range of let's say 1..1024; then it'll automatically time your code while searching for the best sample in the range. I needed to create this for the next item, which is
- My C# Thread tool which runs managed threaded code for you. It creates no garbage so and uses the above optimizer on three different variables so as to find the best amount of threads, calls and inner loop sizes to use for your code. Because of that it can run managed code almost as fast as Burst/Jobs (whereas they use their own compiler and optimize code instead of using Mono). It can even beat Burst on small to medium workloads as long as the code inside doesn't contain some efficiency that Burst can natively fix.
- Then of course there are other things an ECS needs like pathfinding, unit collision avoidance etc. For these I have both CPU and GPU solutions since CPU only really works up to about 10k entities at a decent framerate.
So yeah, I don't recommend trying it; it's probably a total waste of time.
I added up a score for myself from that list giving me one point for something that more-or-less matches an item from this list and a half point for something vaguely similar. I got 15.5
The compiler, chess engine, and text-editor have been targets for years but I haven't quite gotten around to doing them.
I like the idea of the bitwise challenge.
I had been thinking that another challenge that I'd like to see (and have a go at) might be a plotPixel Challenge.
Unsure on byte for 256 indexed color or 32 bit for setting 24 bit color from ARGB as r= rA+R, g=gA+G, b=b*A+B.
It would be easy enough to test if a game obeyed the rules because a full video capture of the game should be possible by intercepting just those functions.
Perhaps a (generous) source code size constraint to stop people going too crazy. (maybe 64/128/256k source-code, zipped). Not for code golfing as much as to avoid people trying to squeeze Crysis though a straw.
I'm writing a shell right now in ruby after seeing a few other folks doing so and looking at the current state of shell development and not finding anything I really liked. It is indeed, a lot of fun. Using Vidar's really quite nice text editor to do so is equally pleasurable.
I agree. It’s fun to just write code to solve problems or even for fun. I love go (it’s the best all around language and it’s not even close) so for fun I wrote a game engine and a small game that I might try to turn into an actual game. It’s good to keep fresh. I’ve also tried to do llm free days
I love the article but disagree with the final point. It wasn't until LLMs came out that my toy projects have all been greatly upgraded with features and functionality. I simply didn't have time to do so before.
Problem/Truth is that writing toy software is not profitable - that's not to say one must always work on profitable things; what i mean is that it's being kept unprofitable, because in a hyper-saturated monopolized attention economy, every piece of "toy" software that isn't a video game or a plugin to some proprietary app ecosystem is essentially an act of subversion, an attempted "theft" or piracy of what the Company considers its rightful property: the sum of all disposable human attention.
I share the same feeling. Programming can be an entertaining and joyful activity where enthusiasts don't really need to worry about creating something useful that will be shared with others.
It's like cooking a beautiful and tasty gourmet recipe that serves only one.
gustavo!!! your website was the first thing i thought of as i was reading this blog post. im a huge fan of your teaching style. thank you for everything you do! :-]
Very nice collection of toy projects, I resonate with you, I also wrote a ton of toy projects in sabbaticals, check the better ones out here : https://milgra.com/
Ever since Nintendo let the mask slip and got up to their old evil bullshit with the Switch 2 launch, I decided to dust off the solution my teenage self came up with: I'll just play the video games I write. Even if they're small and amateurish, they're worlds and experiences crafted personally for me. And I don't have to pay a cent for them, let alone put up with "key carts", DRM bullshit, or we-can-remotely-brick-your-console bullshit.
I'm sure I'll keep buying commercial games from companies with reasonable terms (GOG, maybe?). But writing my own is great fun. It'd be nice if someone else enjoys them, but they're mainly for my own enjoyment.
Oh, and I "vibe code" the old fashioned way: with a REPL and some chill tunes.
1) High prices for the console and games, which may be partially explained by inflation, but still, shit's steep. We're back to the early 90s days of scouring pawnshops for deals and carefully considering which few games we really want to buy. Plus, $10 for a sales demo (Nintendo Switch 2 Welcome Tour).
2) Console must be registered online before becoming playable in handheld mode.
3) Nintendo recently released a terms of service update in which they reserve the right to disable your console if they catch you violating acceptable use. Switch 2s found using piracy carts have not yet been bricked this way, only locked out of internet play (like original Switches were), but still...
4) Some Switch 2 games will not get a physical release, only a cartridge which provides an "activation key" for a digitally downloaded game. Why even bother with this bullshit, I don't know, because it's all of the downsides of physical media and all of the downsides of digital downloads, with none of the upsides of either. On a console.
I don't really get a lot of joy out of building copies of things other people have done better than me before. Almost by definition, no list of cool toy projects is going to appeal to me for that reason. I might like the techniques or algorithms employed within them, but frankly if I'm after that kind of knowledge I just find Leetcode and traditional studying to be both higher value and more fun.
But I do like writing programs in some strange new, non-CS domain where very little software to address my very particular needs already exists. Most of the software I've written in the last few years has been of this kind, for the purposes of learning Finnish. It is amazing what a few shell scripts, fzf wrappers, and well-timed calls to an LLM can do for accelerating one's own learning process.
An anecdote about how a toy project _really_ helped me out:
I work on Dart professionally. I've also been tinkering on a toy programming language that may never see the light of day. My toy language has gone through several incarnations but a while back I was working on adding algebraic datatypes, pattern matching, and exhaustiveness checking to it.
When I first started dabbling in programming languages over a decade ago, I tried to understand that paper for weeks and just could not wrap my head around it. I don't have much of a formal CS background and that paper was impenetrable to me.
So here I am tinkering on my toy programming language and I run into again. I give it another try and laboriously implement it in my interpreter, basically doing a straight translation. I don't understand the code, but it seems to sort of work. So then I start refactoring it a tiny bit at a time into a style that fits the way I think about code. Eventually the algorithm sort of folds into itself and before I know it, I finally understand how it works. The end result was only a page or so of code, but my toy language had real exhaustiveness checking work.
Meanwhile, at work, I am working on adding pattern matching and exhaustiveness checking to Dart [1]. Exhaustiveness checking here is a much harder proposition because Dart has subtyping, unlike my toy language and the ML languages that Maranget's paper works on.
I'd been hacking away at an exhaustiveness algorithm for Dart based on a couple of papers about Scala's approach but they were sort of approximative and inelegant (in my opinion, maybe they are a great fit for Scala).
But once I understood Maranget's algorithm from implementing it in my toy project, it finally clicked for me how it could be adapted to work with subtyping in a sound, coherent way. I wrote it up as quickly as I could and (with a lot of additional help from a teammate to handle generics), that became the algorithm we shipped:
I love this, this is exactly the sort of thing I've experienced too: building the toy project gave me an on-ramp to something similar but much more complicated in the future.
I'm always mixed on this. In the moment it can be fun. In the end when it inevitably feels useless, idk it feels bad. Like I spent a ton of time on something and get some knowledge out of it. It doesn't make my job better and I guess doesn't improve my situation much. My rust sub-directory feels like a digital closet of rpis collecting dust.
The things I find fun writing and learning about and the tools I use to do it are so disconnected from anything i do professionally.
>Perhaps you’re a user of LLMs. [...] But I might suggest resisting the temptation to use them for projects like this.
I agree. However: Your best bet is likely to build it by hand and then see if you can get an LLM to build it.
It is looking like we are living in a world where an important tool to have in your toolbox is going to be the skills and experience of working with the LLM/AI tooling. There are skills, knowledge, tools, tips and techniques that you need to develop to effectively use these tools, just like the honing you get from writing the code.
Over just the last half hour I did a brain dump of "hot tips" to a friend of mine who is just starting to use Claude Code to kick-start him.
The LLMs are REALLY good at writing small toy programs, and at the end of the day it's great to have little tools that you'll use every day.
Good list of projects. I have to disagree on the point on usage of LLMs.
These projects are going to be sideprojects and with busy schedule it might get tough to find motivation. Using LLMs for some of the parts can give the momentum required to continue.
We can also learn from studying the output generated by LLMs.
it's more of compression. fastest way to get 10 yrs of backend sense in like 3 weekends is build fake OS scheduler or just kv store with fs persistence. and the shortcuts your brain takes after that, scary fast. you just start seeing stuff everywhere. like oh this LRU cache is just dumb DLL with hashmap. or this parser is just recursive descent with some peek-ahead logic stapled on. and then you can't unsee it. you start breaking down every abstraction you touch. even prod libs look like toys.
also lol toy projects expose your defaults hard. how you name vars when there's no PR. how you layout dirs when no one's looking. do you even test stuff or just printf until it works. do you log or do you trace or just guess.
I’d triple the estimates, really. For example a game boy emulator is not trivial; there are far too many edge cases with a zilog. Chip8 would better fit for that time frame.
It seems to be an unpopular opinion here. But the last thing I want to do when I get off of work is touch a computer. I haven’t desired to be a hobbyist developer nor have I written a single line of code that I haven’t gotten paid for since the day I walked across the stage with my CS degree in 1996. I was hobbyist for 10 years before then.
So you don't look like sh* that's more impressive than cranking out code that absolutely no one cares about. Especially things that have been done 100s of times.
There's ZERO room for innovation in the projects suggested.
Wanna do some creative thinking? Read some books, learn how normal people think. Explore latest datasets. That's where the meat is. 98% of what you read online has been written by super uncreative(boring) people. They have no imagination whatsoever, IQ is well statistically speaking not too flattering.
Internet is ran by the bottom. It's slop by the bottom, for the bottom. So you need to hang out with creative people.
Tech people write the most boring blog posts because it's aimed at getting hired. Nothing of substance is shared. That's why gwern stands out like a sore thumb. The majority of human beings are incapable of creative thought.
> There's ZERO room for innovation in the projects suggested.
I do not agree. Improved variants and innovation would be possible with operating systems, GUI, programming languages, etc. It would also be possible that something will be made better for one purpose because others are not as good for some purposes, but it is not the only way to do, though.
That philosophy has some lingering halitosis of truth to it.
Any inspection of any economy suggests otherwise. How many tire manufacturers are out there? How many different brands, styles, and types of tire? How many total US Dollars per year are those companies sharing?
How about oil changes? Car washes? How about car wax?
New doesn't equal profitable. Making a different product, or service, is where most of the profits on earth sit.
Wanna be a rebel and broke? Then by all means follow the new-new and barely break-break even.
> Perhaps you’re a user of LLMs. I get it, they’re neat tools. They’re useful for certain kinds of learning. But I might suggest resisting the temptation to use them for projects like this. Knowledge is not supposed to be fed to you on a plate
Am I the only one using LLMs as if they were a search engine? So before LLMs I was searching on Google things like "pros cons mysql mongodb". I would read the official documentation of each db, forums, blog posts, stackoverflow entries, etc. It was time consuming on the searching side. The time it took to read all the sources was fine for me (it's learning time, so that's always welcomed). Now with LLMs, I simply prompt the same with a little bit more of context "pros and cons of using mysql vs mongodb when storing photos. Link references". So, I get a quick overview of what to keep an eye on, and the references are there to avoid relying on hallucination.
It's true that sometimes I go ahead and say "give me a data schema for storing photos metadata in postgres. I wanna keep X in a different table, though" (or something like that). But I do that because I know very well what the output should look like (I just don't wanna spend time typing it, and sometimes I forget the actual type that I should use (int vs integer?)).
The few times I've used LLMs as question answering engines for anything moderately technical, they've given subtly-but-in-important-ways incorrect information such that taking them at face value would've likely lost me hours or days of pursuing something unworkable, even when I ask for references. Whether or not the "references" actually contain the information I'm asking for or merely something tangentially related has been rather hit or miss too.
The one thing they've consistently nailed has been tip-of-my-tongue style "reverse search" where I can describe a concept in sufficient detail that they can tell me the search term to look it up with.
Absolutely. And I’m finding the same with “agent” coding tools. With the ever increasing hype around Cursor I tried to give it a go this week. The first 5 minutes were impressive, when I sent a small trial ballon for a simple change.
But when asking for a full feature, I lost a full day trying to get it to stop chasing its tail. I’m still in the “pro” free trial period so it was using a frontier model.
This was for a Phoenix / Elixir project; which I realize is not as robustly in the training data as other languages and frameworks, but it was supposedly consuming the documentation, other reference code I’d linked in, and I’d connected the Tidewave MCP.
Regardless, in the morning with fresh eyes and a fresh cup of coffee, I reverted all the cursor changes and implemented the code myself in a couple hours.
>The one thing they've consistently nailed has been tip-of-my-tongue style "reverse search" where I can describe a concept in sufficient detail that they can tell me the search term to look it up with.
This is basically the only thing I use it for. It's great at it, especially given that Google is so terrible these days that a search describing what you're trying to recall gets nothing. Especially if it involves a phrase heavily associated with other things.
For example "What episode of <X show> did <Y thing> happen?" In the past, Google would usually pull it up (often from reddit discussion), but now it just shows me tons of generic results about the show.
Yes, you have to be very careful when querying LLM's, you have to assume that they are giving you sort of the average answer to a question. I find them very good at sort of telling me how people commonly solve a problem. I'm lucky, in that the space I've been working has had a lot of good forums training data, and the average solution tends to be on the more correct side. But you still have to validate nearly everything it tells you. It's also funny to watch the tokenization "fails". When you ask about things like register names, and you can see it choose nonexisting tokens. Atmel libraries have a lot of things like this in them
#define PA17_EIC_LINE PIN_PA17A_EIC_EXTINT_NUM #define PA17_EIC_BIT PORT_PA17A_EIC_EXTINT1 #define PA17_PMUX_INDEX 8 //pa17 17/2 #define PA17_PMUX_TYPE MUX_PA17A_EIC_EXTINT1
And the output will be almost correct code, but instead of an answer being:
PORT_PA17A_EIC_EXTINT1
you'll get:
PORT_PA17A_EIC_EXTINT_NUM
and you can tell that it diverged trying to use similar tokens, and since _ follows EXTINT sometimes, it's a "valid" token to try, and now that it's EXTINT_ now NUM is the most likely thing to follow.
That said, it's massively sped up the project I'm working on, especially since Microchip effectively shut down the forums that chatgpt was trained on.
Try perplexity — I found it to be very good at digging up information. It became a nearly complete replacement for web searches at this point.
This. I was skeptical at first, but it is indeed good at searching and answering questions without! That said, I still have to double-check results for niche queries or about stuff that is relatively new. Sometimes, the "sources" for the answers are just someone's opinions — unsubstantiated by any facts — on an old Reddit post that's only tangentially related to the topic. And sometimes, you simply know that manual search and digging through SO answers yourself will yield better results. At this point I've developed a gut feeling that helps me decide whether to prompt Perplexity or just g**gle it.
> the "sources" for the answers are just someone's opinions — unsubstantiated by any facts
Isn't that the nature of the web?
I mean, that's exactly what I expect from web searches, so as long as you don't consider the fancy-looking [1] citations "scientific", it just digs up the same information, but summarized.
I don't expect miracles: Perplexity does the same things I've been doing, just faster. It's like a bicycle I guess.
It's my default search engine.
It is annoying when I really do want just a link. In that case, I would tab select the Google search engine in Chrome.
But for a lot else, Perplexity very nice.
It's interesting because I use them every day all day for this now.
You have to "gut check" the answers and know when to go deeper.
A lot of answers are low stakes, and it's OK to be a little wrong if it helps go in the right direction.
I agree. Use with caution. One of my personal pet peeves with LLM answers is their propensity to give authoritative or definite answers, when in fact they are best guesses, and sometimes pure fantasy.
I was trying to set up a solr cluster on kubernetes the other day, and was googling how to create a new collection.
Google AI helpfully showed me this awesome CRD that created exactly what I wanted... sadly, there is no such CRD in reality
[dead]
> Now with LLMs, I simply prompt the same with a little bit more of context "pros and cons of using mysql vs mongodb when storing photos. Link references".
In near future, companies will probably be able to pay lots of money to have their products come up better in the comparison. LLMs are smart enough to make the result seem "organic" -- all verifiable information will be true and supported by references, it will only be about proper framing and emphasis, etc.
This was already a problem in a world without LLMs. Reputation is the only human mechanism that mitigates this.
Technology is a catalyser. It has a knack of turning things from "already a problem" to "now a catastrophe".
"To err is human, but to really foul things up you need a computer." -Paul Ehrlich
Now LLMs will create your bugs for you!
It's seriously exacerbated by smug LLMs.
I'd say LLMs have helped a lot with this problem actually, by somehow circumventing a lot of the decades of SEO that has now built up. But, I fear it will be short-lived until people figure out LLM optimisation.
I'm very grateful that we have a lot of players training LLM's, including several that are published as open models and open weights.
I fully expect LLM results to start including ads, but because of the competition I hope/believe the incentives are much better than they are for, say Google's search monopoly.
It could potentially be more insidious though.
We'll probably start sending prompts to multiple models and comparing the results with lower-power local models.
I start with "I don't have any money, I need an open source library that does this" lol.....even at work.
Open weights don’t solve it; the promotion will happen during post training and the results will be in the weights.
I really hope that they don't include ads in paid tiers. But I'm not sure how much you would actually have to pay to cover the potential lost ad revenue... it might be too much.
> LLMs are smart enough to make the result seem "organic"
I would never describe the output I've seen from LLMs as "organic".
Would be funny if LLMs get SEO'd to death and go the same way as search engines
This would be the Kessler Syndrome of the Internet, and it will be glorious.
This is already the case, SEO content, sponsored comparison sites, influencer marketing, it's all about subtle framing. LLMs just supercharge the problem by making it easier and cheaper to scale.
The real issue isn't that LLMs lie, it's that they emphasize certain truths over others, shaping perception without saying anything factually incorrect. That makes them harder to detect than traditional ads or SEO spam.
Open-source LLMs and transparency in prompt+context will help a bit, but long-term, we probably need something like reputation scores for LLM output, tied to models, data sources, or even the prompt authors.
They’re already talking about llm search optimization as the new SEO.
I believe the term is Generative Engine Optimization (GEO).
Shit. That's the name of my org. (Geo-spatial data)
This already happens unintentionally, e.g. Wikipedia loops, where bad info on Wikipedia gets repeated elsewhere, and then the Wikipedia article gets updated to cite that source.
When LLM-generated content is pervasive everywhere, and the training data for LLMs is coming from the prior output of LLMs, we're going to be in for some fun. Validation and curation of information are soon going to be more important than they've ever been.
But I don't think there'll be too much intentional manipulation of LLMs, given how decentralized LLMs already are. It's going to be difficult enough getting consistency with valid info -- manipulating the entire ecosystem with deliberately contrived info is going to be very challenging.
tbf, Google has pretty the monopoly on search engines, the rest is way behind (even after enshitification). LLMs seems to be a more competitive space.
Kagi is already far better than Google. It doesn’t effect me if Google gets all the traffic.
I use it the same way. The feeling is that I'm back in ~2010 when Googling stuff felt like a superpower. I could find anything back then.
Of course, it didn't last long, and trying to Google now is an exercise in pain and frustration. Lots of people have complained about the various things Google and marketers have done to get there, idk, I just don't like how it works now.
Top LLMs feel amazingly good at rapidly surfacing info online, and as I go through the references they're usually pretty good. I guess the same forces as before will apply, and there will be some window of opportunity before it all goes away again.
I wonder when LMMs and services like chatgpt become as bloated as search engines are today, with their own equivalent of SEO/SEM tools and other unwanted stuff distracting and disturbing accuracy, even if one finally stops hallucinating.
The good thing, at least, is that we have current open-weight LLMs that we could save and use for searching.
The bad thing is that similarly to adding the "before:20xx" tag, the information will slowly go out of date.
Hopefully not that fast, but I'm pessimistic. The cost of the human bloat will far surpass the current cost of hallucinations. And like we saw with Google, that bloat can become a feature of the content itself, not just contained in the tool.
If you do a web search and find a random blog post full of spelling errors and surrounded by ads, you're not going to trust that at the same level as a Stack Overflow post with a hundred upvotes, or an article with a long comment thread on HN.
But an LLM digests everything, and then spits out information with the same level of detail, same terminology, and same presentation regardless of where it came from. It strips away a lot of the contextual metadata we use to weigh credibility and trust.
Sure, you can follow references from an LLM, but at that point you're just using it a fuzzier form of web search.
> Sure, you can follow references from an LLM, but at that point you're just using it a fuzzier form of web search.
That's exactly the superpower, isn't it? Web search without having to dig deep around (especially now that Google's quality has declined over the years). Sure thing, LLMs are capable of more.
I agree and am perfectly happy using it as fuzzier web search, because it works really well for me.
Finding references is often my main goal, but other times I just want some quick facts, in which case I'll be as thorough with checking as I would when reading a random blog with spelling errors.
Actually, someone should design one that can pull in quotes. Like as a separate tool the LLM uses to quote that has guarantees, it's just a copy and can't be hallucinated. Then you could see the primary info when it's needed/asked for, similar to articles
There will be a race between the attempts monetize online LLM services like this and the development of consumer owned hardware that can enable local LLMs with sufficient power to deliver the same service but ad free.
Combined with RAG a self hosted LLM will definitely be able to deliver a more impartial and therefore better solution.
I don't think anyone can make a Google today that works as well as it did back then. Google shaped how new content was created, and that was probably a much bigger deal than any changes to the tool itself
Sure, but in the context of this thread where the usage case of modern LLMs was described as:
"Now with LLMs, I simply prompt the same with a little bit more of context "pros and cons of using mysql vs mongodb when storing photos. Link references"
locally hosted LLMs with RAG will absolutely be able to do this, better than Googling even back then could, and so the prospect of monetized LLMs with ads in them degrading the user experience for this sort of usage case is unlikely.
I do this. But the killer usecase for me is writing all boilerplate and implementing some half-working stuff keeps my attention on the issue which makes me able to complete more complex things.
A recent example is when I implemented a (Kubernetes) CSI driver that makes /nix available in a container so you can run an empty image and skip a lot of infra to manage.
I talked to it a bit and eventually it wrote a Nix derivation that runs the CSI codegen for Python and packages it so I could import it. Then I asked it to implement the gRPC interface it had generated and managed to get a "Hello World' when mounting this volume (just an empty dir). I also asked it to generate the YAML for the StorageClass, CSIDriver, Deployment and DaemonSet.
So LLM left me with a CSI driver that does nothing in Python (rather than Go which is what everything Kubernetes is implemented in) that I could then rewrite to run a Nix build and copy storepaths into a folder that's mounted into the container.
Sure implementing a gRPC interface might not be the hardest thing in hindsight, but I've never done it before and it's now a fully functional(ish) implementation of what i described.
It even managed to switch gRPC implementations because the Python one was funky with protoc versions in Nix(Python bundles the grpc codegen it's so stupid) so i asked it to do the same thing for grpclib instead which worked.
I hear a lot of people talk about LLMs writing the "boilerplate" and wonder why they haven't abstracted that away in the first place.
Maybe my brain has been permanently altered by hacking Lisp.
The problem with Lisp (or at least Clojure) is that abstracting away the boilerplate requires you to correctly identify the boilerplate.
It’s nontrivial to structure your entire AST so that the parts you abstract away are the parts you’re not going to need direct access to three months later. And I never really figured out, or saw anyone else figure out, how to do that in a way which establishes a clear pattern for the rest of your team to follow.
Especially when it comes to that last part, I’ve found pragmatic OOP with functional elements, like Ruby, or task-specific FP, like Elm, to be more useful than Clojure at work or various Lisps for hobby projects. Because patterns for identifying boilerplate are built in. Personal opinion, of course.
Yes, good tooling shouldn't have boilerplate. Minimizing loc (within reason, not code golf) is the best thing you can do for maintainability. Unfortunately things like Java are popular too.
I hear you. But removing boilerplate via abstraction (Lisp) is very different from generating it on demand (LLMs). The former is obviously qualitatively better. But it requires up front design, implementation testing etc. The latter is qualitatively insufficient, but it gets you there with very little effort plus some manual fixes.
> The latter is qualitatively insufficient, but it gets you there with very little effort plus some manual fixes.
I remember years ago, when I worked at a large PC OEM, I had a conversation with one of our quality managers -- if an updated business process consumes half the resources, but fails twice as often, have you improved your efficiency, or just broken even?
"Qualitatively insufficient, but gets you there" sounds like a contradiction in terms, assuming "there" is a well-defined end state you're trying to achieve within equally well-defined control limits.
But LLMs can help with the former too.
Boiler plate that is primary (not generated from a concise generation) is counterproductive: instant tech debt.
The way to to use AI is to get help writing that generation logic, not to just get it to crank out boilerplate.
You're not winning just because AI is taking the manual work out of cranking out primary boilerplate.
There’s necessary complexity like error handling, authz, some observability things, etc. which can’t be trivially abstracted away and needs to be present and adjusted for each capability/feature.
Boilerplate can be a slog to write but a breeze to read.
i’ve stopped writing “real” code for the most part, i just bang out some pseudo code like:
and tell it to translate it to x lang.so i control the logic, it handles the syntax.
asking it to solve problems for you never seems to really work, but it remembers syntax and if i need some kinda reader interface over another or whatever.
can’t help me with code reviews tho, so i spent most of my time reading code instead of remembering syntax. i’m ok with it.
It can solve problems, as long as they’re practical, or things done before.
LLMs are more advanced than this as shown by my example, I don't think you'll find many CSI drivers in Python on the internet.
yeah, that’s what works for me also. LLMs are a nightmare for debugging but a breeze for this.
another good use case: have it read a ton of code and summarize it. if you’re dealing with a new (to you) area in a legacy application, and trying to fix a problem in how it interacts with a complex open-source library, have the LLM read them both and summarize how they work. (while fact-checking it along the way.)
> Am I the only one using LLMs as if they were a search engine?
Nope, you're not alone as I also do this. I'm also not using any AI IDE's (yet).
It's funny, I actually recently failed a live technical interview where I was using my LLM of choice to answer Google-like queries instead of an IDE like cursor. The interviewer told me that he had actually never seen anyone use AI for coding like that before. Up to that point, I assumed that most coders were mainly using AI as a search engine and not necessarily using AI IDE's yet. Are we really that rare?
Not at all, I've been doing this with ChatGPT and Claude for a long time. I only recently (last couple weeks) started playing around with Claude Code on command line (not in an IDE). I didn't like Cursor very much. YMMV
In the past you looking around documentation, SO answers etc would have hopefully helped you learn more about the tools and develop skills required to independently analyze the pros and cons. If you ask an LLM (or a search engine or a colleague) and take their words as the ground truth then you won’t develop the skills. Worst, sooner or later no one will have enough knowledge nor analytic skill to form an opinion on any sufficiently deep subject and they will all be dependent on corporate chatbots to spoon feed them information that may or may not be biased in a way against your interest. Now imagine if ChatGPT tells you to use azure and Gemini tells you to use GCP…
This, and I use them for code review and rapidly generating prototypes that I heavily edit. Almost none of the LLM code survives usually. You could ask "Why dont you just write it yourself then?" but sometimes getting started with the skeleton of a working project is the most difficult part of it.
It is nice when it works, but sometimes I run into trouble where I don't know a right word to put in the prompt to get the answer I'm looking for. I've recently been playing around with Raku and had a really cryptic type signature error and Claude was of absolutely no help because it didn't know about the interaction of Signature literals and 'Slurpy' sigils in method parameters. Only when I learned about it and included the word Slurpy in my prompt would it actually regurgitate the information I was looking for but at that point I already knew it.
To be fair, this isn't a new problem - many times I've wanted to find something but not even known what to Google for.
I think the key difference here is that if you type into Google the wrong thing it will return poor results that make it fairly clear that you're not on the right track
LLMs will sometimes just invent something that basically gaslights you into thinking you're on the right track
This plus easier search of poorly/not-at-all documented APIs is like 80% of my usage too. Besides that, a lot of “here’s my design for xyz system, am I a stupid idiot for using this architecture?”.
Yup this is where 90% of the productivity benefits come from for me. Instead of needing to spend an hour scouring documentation I can ask an LLM and have an answer in 5 minutes
> I would read the official documentation of each db, forums, blog posts, stackoverflow entries, etc. It was time consuming on the searching side. The time it took to read all the sources was fine for me (it's learning time, so that's always welcomed).
This learning time that you welcomed is what you will now miss out on. The LLM gives you an answer, you don't know how good it is, you use it, and soon enough, if running into a similar issue, you will need to ask the LLM again since you were missing out on all that learning the first time which would have enabled you to internalize all the concepts.
It's like the regex engine example from the article. An LLM can create such a thing for you. You can read through it, it might even work, but the learning from this is orders of magnitudes less than what you get if you build this yourself.
I think it depends. LLMs can link references of where they took the content they throw at you. You can go and read such references. I like what LLMs provide, and at the same time I don't wanna blindly follow them, so I always allocate time for learning whether it's with LLMs or not.
Only if you use LLMs wrong. Today's models have deep research which will generate a comprehensive analysis with proper citations
I feel like I should point out that's the dialog engine not the model itself.
Yes, I think that is understood by everyone
You'd be surprised. A number of fairly technical people who are just not that familiar with ML I know got confused by this and believed the models were actually being tuned daily. I don't think that's universally understood at all.
That has actual practical implications and isn't just pedantry. People might like some model and avoid better dialog engines like perplexity believing they'd have to switch.
I meant "everyone" in the context of HN ;)
sorry but you’re underestimating the number of people who come here, and the range of backgrounds (and interests) they have
I think you either didn't read my response or missed the point. No matter if the LLM output is useful or not, the learning outcome is hugely impacted. Negatively.
It's like copying on your homework assignments. Looks like it gets the job done, but the point of a homework assignment is not the result you deliver, it's the process of creating that result which makes you learn something.
Best metaphor I have found to how I use them is as "a hunting dog".
They can get into small crevasses and the foliage and whatnot and they don't mind getting wet. They can fluster rabbits out. And are somewhat smart. But you still have to make the kill and you have to lead the dog, not the other way around.
"Copilot" is a great marketing name, but a bit deceiving.
For really basic overviews of different technologies I've found YouTube videos to be useful.
For code examples I've always found GitHub code search to be super useful.
I haven't really found a use case for LLMs where it's any faster than the research tools already available on the internet.
A good example is writing AWS CDK code, which is a pain in the ass. But there are so many examples to pull from on GitHub code search that are being used in real projects that I've found this method to be faster than prompting an LLM that may or may not be correct.
I just used an LLM to sleuth out a concurrency issue earlier - I could see what was happening in the debugger, I just couldn’t really see why - asked the LLM to talk me through lifecycle stuff, and boom, first thing it brought up was apparently the answer. Thank you, glorified fuzzy search! Much quicker than pouring through the docs.
I got lucky sure, and sometimes I don’t get so lucky - but it works often enough to have become a part of my regular approach at this point. Why is the thing broke? Maybe the robot knows!
Even when using them to code, I use them as a search engine. Rather than telling them to implement feature X, I clone a repo which has a similar feature and say:
"explore the repo I've cloned at /src/foo and explain how it achieves barFeature. Now look a the project at /src/baz and tell me why it would be difficult to use foo's approach in baz"
I rarely have it do anything novel, just translate ideas from existing projects into mine. Novel work is for me to enjoy coding directly.
> Am I the only one using LLMs as if they were a search engine?
Do you often find the need to search for APIs that do not exist but wish they did, and what the code using these imaginary APIs might look like?
I learned about window functions in SQL using an LLM. I hadn't written SQL in over a decade and never ran across them. It explained how they work and the trade-offs. It was great!
My main use for LLMs is to ask it to give me some boilerplate configuration for some library/tool I am not very familiar with and then look up the docs for the options it spits out. Like:
"give me the terraform configuration for an AWS ECS cluster with 2 services that can talk to each other, with one of them available publicly and the other one private"
Occasionally to give some small self-contained algorithms, for example:
Give me a way to format the difference between dates as human-readable text. For example: 1 day, 16 hours.
> Am I the only one using LLMs as if they were a search engine?
Google agree with you and now at the top there is a AI generated answer, clearly labeled as AI generated, and it cites the sources. I was trying to escape AI, but I have to recognize the implementation by Google is quite good.
Perplexity Pro is even better. True, it costs $20/month, but you don't have to look at that dog's breakfast of Google's homepage...
> Am I the only one using LLMs as if they were a search engine?
I quite like the search-first LLMs like Gemini and Copilot for this reason. They give you links which you can use to verify the output and seem to be less prone to directing you to SEO spam than Google Search/Bing.
Although I do partly think that search engines today are delivering poor performance compared to what they delivered historically so LLMs are benefiting from that.
I primarily use it this way or as a rubber duck. I pretty much use it as a replacement for going on a Google adventure. When I use it for code, it's mostly as a glorified auto-complete or write some boilerplate I can't be bothered to type.
It's kind of scary how good it is. I haven't completely switched over, but I think a lot of that is just not wanting to admit that this is the new paradigm and the implications. The ability to not only find what you're looking for faster, but have it tailored to your specific context? It's hard to go back from that.
i'm currently using it in large part as a question asking tool. feed it a bunch of documents, then ask it to ask me questions to figure out whats been missed -- stuff that i probably know, or could find out, or look for more documents that have the answer, but arent clear from the documents i had handy.
this afternoon i spent some time feeding context, with the idea of making a QnA context for maybe 50 different website routes, so i can have it ask me similar questions per route, so i can have a hell of a lot of pretty accurate documentation for an intern to pick up a couple to work on, or get some LLM to take a try at filling out the details.
im feeding it knowledge, with the output being documents, rather than me getting knowledge from it.
My most common use is to treat it like one of those recipe clippers that grabs a recipe out of an overly long blog post. Last weekend I punched '28 years after credits?' into Kagi and got back only what I wanted. Finally I can have a recipe clipper for any topic. I'm happy with it.
That's basically how I use it though I cannot wait for Gemini to be a click away in my browser with the current screen/page(s)/tab(s) embedded so I can ask it stuff about the current long article/documentation page. We're becoming very, very lazy.
I tell to myself to use them as "teachers", not "interns", i.e. ask them questions to guide my process or look for the sources of knowledge needed to understand or do something, instead of asking them to get things done (except tedious, simple tasks).
Make sure to spend some time asking them questions on topics you already know a lot about. (I like to ask the AI about a game I developed called Neptune's Pride.)
A year ago the AI would just make up some completely random stuff. The current crop do a very good job, but still not 100% correct.
They are wrong enough that I would be wary of using them to "teach" me topics I don't know about.
> Am I the only one using LLMs as if they were a search engine?
That's basically how I use them.
Nope, I do this too (most of the time.) I don’t like working on code I don’t understand. I have started to ask it to use a client API I’ve written to figure out how clients would work with the stuff I write though. It’s great.
I also treat chatgpt as analogy to google. works relatively fine.
I think one of the major issues I (and others) have with LLM's as search is they are heavily biased towards outdated data. The other day I was trying to have it get me the most updated versions of X, X, X from an old project. I tried several providers and all of them without exception gave me an "upgrade" to an older version than was in the project for at least 1 item(by years). I'm sure they are choosing LTS releases over the new ones because that is most popular overall in a dataset. However no matter how "hard" I prompt they keep giving me those old ones over "latest"
Cursor is awesome as search engine for your codebase. Extremely helpful if you are not familiar with it.
My LLM use case preference is exactly that - a google replacement. I've modified my prompts to ask for links to direct source material, thereby allowing me to go deeper. I find this in no way different then a search engine.
The downside however, is that at least 50% of the links cited no longer exists. This points to a general issue with LLMs, temporal knowledge. I tend to at least go to archive.org on a per-link basis when so inclined.. But clearly that's not ideal.
Does anyone have a better solution, short of just asking the LLM to remove the extra step, and link to the archive.org cache for the ingestion date (which frankly, I also don't have faith in being accurate).
Same. By appending a ? to a query, Kagi returns a summary with references. Quick and easy.
This work great for me too, when learning something new the LLM brings in all sorts of perspectives and contexts I don’t know about yet.
It pulls in info from the docs, forums, code samples, shows me how things are connected, and may even suggest why something is good or bad.
Plenty of opportunity to look up docs I didn’t even know about yet.
It's a suggestion engine. An autocomplete (I am not saying GPT is just autocomplete of course it is more advanced) but end if the day you need to verify everything it does right now. May not be true in the future though.
Search finds you sources then you can decide if you trust them. AI generates code and you have no idea if what it generated was perfect, slop or almost perfect but bad in some major way.
I use perplexity when I want sources for my info, chatgpt if I want a quick overview.
Well now the search engines have LLMs too, which are often useful, even if only to avoid cookie popups.
Nope! I use llms like I used to use Google and StackOverflow. Really great to bounce ideas off of, brainstorm, yes save time too ... But the at the end of the day, what will separate programmers in the future is those that bother to put an ounce of effort into things, read, and learn. Versus those who want the easy button and answer given to them. They're going to be the next generation of people who leak your passwords, social security numbers, PII, and create massive global security incidents.
To be clear. AI doesn't kill people. People kill people. In this case, lazy people. Welcome to Idiocracy my friends.
One of the best things that I’ve done for myself, career-wise, came during a 6-month sabbatical between jobs. I had a bunch of projects that I wanted to do, but I kept finding myself stymied by scope creep - when you don’t have any constraints, it’s hard to decide how big a project should be, and the projects would grow to these massive unfinishable beasts.
I decided to constrain myself to one week per project - whatever I could get done in a week was what I’d do for this project. The experience of going from zero to something useable in either a new language or a new framework or a new field in a week was enormously confidence-building - learning that I was actually pretty good at this whole programming thing and breaking through a bunch of walls I’d run into previously in my professional career gave me an incredible amount of trust in myself to be able to take on new challenges that really helped me when I was looking for a new job. At the same time, it reminded me what I loved about programming - in a week, I was able to build something that solved a problem I had or that someone I knew had, and I got to do so in a creative and intellectually challenging fashion.
If ever you find yourself in a position to take a couple months off between jobs, skip the leetcoding or whatever else and just build some toy projects - you’ll be amazed how much you already know.
I would just add that gen AI can actually be a fantastic aid when you're building some toy projects just for yourself.
I am primarily a backend engineer, though I am capable on the front end, just not particularly fast. I am especially slow at the voodoo that is CSS. In the past, I would have been turned off from doing some end-to-end personal projects because I know I would have been bogged down just battling CSS for hours and hours (it's particularly problematic that I'm slow at CSS but also a stickler for how I want things to look). With AI tools I can literally just say "make it pretty". It gets me about 85% of the way there, but importantly it gets me to the point where it puts in all the styling framework and then I just have to fix bugs/edit/tweak it, which goes very quickly.
So now I've found myself much more likely to build personal projects because I know I can get past things that in the past felt a bit like quicksand to me.
It's a balance I find.
On the one hand, the machines can spin up these bridges over unfamiliar code chasms.
But on the other hand, you (at least I) have no investment in the code. No familiarity with it. No "skin in the game" when I can tell the AI to just redo it.
One way around that is by using the "Chat bots as search engine", and then, rather than cut and pasting stuff into your project, type it in by hand. Even if you're just straight up copying it, by typing it in you have a bit more connection to it. When I do that I can better see the unfamiliar patterns rise out of the code. It's much better for me than just reading it.
Also gives me opportunity to tweak it as I like, which can break things, which can teach me things.
Mind, I've been advocating this "type it, don't paste it" philosophy for years and years, for any code snippet found in a book or tutorial or wherever. It's just now with the bots, the code is more tuned to our exact use case, and the incentive for moving it over is less.
But I find grabbing a blob of code, pasting it into my project, and getting errors so that I can tell the bot whats wrong is not particularly satisfying.
Watching Claude figure out its own errors even less so. (It's very cool to watch, just not very joyful to me outside of "wow, that's cool".)
On the other hand, I have no problem pasting a snippet back with a "what exactly is this doing here" to get better explanations of what I'm seeing.
> rather than cut and pasting stuff into your project, type it in by hand
This type of mechanical copying is probably very important for learning and staying familiar with what it is doing.
Just last night I had ChatGPT help me with some algebra and geometry for a hobby project I'm working on. I'm rusty and got stumped so I asked the bot to unblock me. It helpfully walked through the entire simplification / substitution / solving process step-by-step, which I manually copied down using pencil and paper. Following along like that, mechanically, really helped it click for me. I plan to do more of that.
Hard agree on this.
I've come to the point of using current AI tools as "search on steroids". For coding tasks, I now ask questions using more generic language, looking for references, and when the AI comes back with specific examples, I try to refine the question in terms of the example that the AI has given me. "Kinda like that, but inside of Book, I actually want to do X with the Author"
This makes it so that I have to manually translate code that the AI is showing me to my specific case, typing it out myself in my code's context. This has helped immensely with developing my familiarity with a couple of new (to me) frameworks, and with slowly adding to my overall understanding of what I'm /actually/ building.
The joy of personal projects is they can look like shit! You'd also be surprised how much you can learn by just screwing around with something for a day. A big part of the whole exercise for me was getting out of my comfort zone and learning that I can learn. I'm primarily backend as well, and came up through the ops side (as opposed to formal CS), which always led to a lot of imposter syndrome, so a lot of the benefit came from just putting myself in an uncomfortable position and just staying there long enough to work my way through it.
That said, my goal for this was to wear the hair shirt (and also this was before GenAI was really a Thing) - if you just want to make something neat, absolutely vibe your way there.
But I don't want them to look like shit.
When I have gen AI create a starting point for the CSS for me, I still go over every line, and I always have to do a bunch of fixes and edits to get this to look how I want. Frankly, dicking around with the esotera of CSS is something I absolutely don't want to do and don't care about - it's really just reminding me of syntax that I forget because I don't do it often enough. This is also why I find gen AI invaluable for shell scripting. Hunting through reams of man pages doesn't make me learn any faster than just looking at the specific arg flags gen AI outputs (and potentially asking AI more about the details of that command). I just don't do enough shell scripting to really remember the command/arg specifics long term for things I don't commonly do.
> So now I've found myself much more likely to build personal projects because I know I can get past things that in the past felt a bit like quicksand to me.
I'm the opposite of you (my strength is in voodoo) but my experience is the same. Whereas before I'd spend 2-3 weekends getting bogged down on elements that aren't my strengths and never complete, now I can easily whip up a mostly functioning web app (just for me) in just a few hours.
This is how 100% of non-technical executives view software engineers, really. And probably explains a lot of the hysteria (unfounded as I may believe it to be) around LLMs eating the industry.
A good reality check is: if a stranger asks you about a specific part of your toy project, would you be able to explain it?
If you can't, it means there's something there you don't understand, and you lost an opportunity to learn. Of course, this is also true for reusing libraries and stuff.
Within a job, what matters is the _team learning_, and it's roughly the same idea. At least one person in the team should be able to explain anything the team does. Trusting it to an AI puts the whole team in a precarious situation, even if it is something as simple as a CSS layout.
I would argue that, at least for me, gen AI helps me learn things much faster.
E.g., like I said, previously I would have been turned off from doing some projects because I wouldn't have spent a ton of time dicking around with things that are just inefficient. Now I have gen AI create a starting point, and it's very clear to me "Ah, this is how I would use grid layout to get what I want". I can tweak things and it's very clear to me how things are working.
>A good reality check is: if a stranger asks you about a specific part of your toy project, would you be able to explain it?
I like that as a benchmark. I think it also works with LLMs too. I have had best results with AI generation of code when I pass that check. When using a LLM, the chatbot is the stranger.
In fact much of the frustration I have with using AIs seems to be for models tuned to the situation where the user does not have the ability to explain it specifically and it has to make broad assumptions. This may be the best result for a lowest common denominator, but it's frustrating when it provides a generic solution to a specific problem.
I can see specialised models diverging to focus on different levels of assumed knowledge and expectations. Perhaps to have modes of thinking, maybe as a more tone based MOE. Is anyone doing mixture of finetunes?
unless you own/write the whole software stack(not the web stack), i doubt anyone can fully explain their project.
For example, if in your project you have to write a file, and someone ask you how does the kernel cache file writting, that would not be an easy question.
You're missing the point and taking it to an extreme.
Let's say I write a toy project to log all sites I visit to a text file. The goal is to learn browser extensions, not learn kernel cache. There's a clear boundary, and anyone asking me about kernel cache would sound very weird. It's obviously a lot of layers of knowledge deep into the stuff I'm learning.
If my toy project is a beautiful web app, then CSS is _not_ several layers deep. It is in the front of it. It's a dependency I _need to learn_ in order to own my project.
If my toy project is a web app, and doesn't matter if it looks beautiful or not, then I want to spend my time away from CSS (either writing it by hand or interacting with an AI). I am fine with having it ugly.
Understanding these dependencies and layers of knowledge is part of the learning as well.
Again, this works for teams as well. There are certain things, at a certain level, that if you don't have someone in the team that knows it, leads to a precarious situation.
The fact that people don't recognize this, and mistake it for not made here syndrome and other nonsense, is actually scary. It means those recipes (don't reinvent the wheel, good is better than perfect, etc) became mantras that people repeat without actually understand why they exist, and those boundaries have become arbitrary and meaningless.
my current opinion is that uf i can get an ai to write details into a file that another team member's ai can read and then know whats up, thats close to the same as the team member knowing it, as long as theyre either reading the file that the ai read, or working through the ai and providing that file(s) as context.
> If you can't, it means there's something there you don't understand, and you lost an opportunity to learn. Of course, this is also true for reusing libraries and stuff.
Yeah, in libraries that's the aim, software composability and reusability, you don't need to keep everything in your head. Same here.
A "lost opportunity to learn" can be preferable to not making the thing or making it much slower.
I ended up migrating to fixing things in the libraries I use because they piss me off.
Bad onboarding docs, broken or bent SDLC, major performance issues. I spent all day yesterday fixing perf issues in a library that competes with one I loathe for trying to complect an entire problem domain.
You probably have the same thing I do. Teamwork is a very different motivation than working on something either never intended for others or with a long MVP cycle that makes it too easy to get bogged down with side quests. "It would be way less tedious to deal with X if I did Y first. But now Y is blocked so I do nothing."
Versus 'someone is waiting for this.'
In my younger years, classic ASP + SQL was easy, and I had IIS + SQL Server set up on a server at my house. And I could write HTML/CSS/JavaScript.
Deployment was... run update scripts, push ASP files via FTP.
Now I know there are newer ways to do things, and I'd love to set up CI/CD in my personal life, but every time I want to do a "toy project" I get hung up researching and thinking about my own personal software development life cycle.
What you choose for hosting and deployment for your toy projects?
I've got a VPS and domain I've had for a long time - for most of these, docker compose + the VPS did the job just fine. I was in the same boat - I'd fallen behind on a lot of the ops stack, and project setup wound up being a tar pit for a couple projects before I got frustrated and just said "Fuck it" and did it the dumb way, which wound up being the correct approach.
I found the trick with the personal projects was to pick the thing I was trying to learn with that project and focus on that - if I was trying to learn React, don't also try to learn kubernetes or lambdas or anything else. You really have to be comfortable with your project kind of sucking for all this to work.
So I'd say if you want to learn proper modern deploy + CI/CD + etc, take a project where you already know the rest of the stack (hell, make it Hello World if you have to) and use that. Keep things simple and in scope, get something stupid working, and then iterate to make it better.
For most of my toy projects, it's a systemd service file and rsync + restart command for deploys. Might have to SSH in and run an install command occasionally.
You can get a VPS in any cloud provider and treat it as your server, just copy files to it. It depends on the scope of your projects, though, if you need a lot of resources it can get pretty pricey for just toy projects.
I pay $12 for a server on Digital Ocean, and I host multiple servers behind an Nginx. But I am primarily a frontend/Desktop developer, so they are simple.
I am in such a phase as I write this. What should have been a week-long project has already gone through three and not finished. This is a simple multi-platform app I am building in Flutter.
One challenge I face is to be able to build something useful within a week. In two projects, I could make wirhin two days each.
For the Flutter one mentioned above, it took me two-three days just to get the tooling/SDKs to work and get me to a hello-world program working on Windows. Then it took about a week to program with all features I needed.
And then, damned again -- Took me a couple days to get it to build APK (random errors; the issue in the build tooling is still open). And then a week of craziness to sort out Android permissions. I still have the latter unresolved.
Out of curiosity, how did you end up with 6 months off between jobs?
I would love the opportunity to take ~6 months off, but I feel scared that if I just quit, it might take me a longer time to find a job again.
It took me 8 months of active shopping to find a job w/ 25 years of experience. Unless you are really good at interviewing with an interesting resume, I'd suggest caution until the job market recovers.
What do you think the bottleneck was in your search?
Was it finding openings, getting interviews, or passing those interviews?
In my niche I’m finding that there are very few opening in the city I live in, but that getting interviews might not be too hard. Not sure how easy it would be to pass those interviews.
Most of my issues were around interviewing, I got plenty of them but somewhere in the 3+ rounds of interviewing I'd choke. I have always been nervous during interviews and had previously got most of my positions through my network, but none of their companies were hiring. Next time this happens I'm going to try an interview tutor to see if that helps and probably take some Xanax or something to help settle my nerves.
Take it or leave it - I tell myself i already failed the incomming interview and whenever I notice i get nervous I tell myself im just talking with friends
Thanks.
I'm single, rent controlled, and was working at a FAANG-ish company.
The fear was real, for sure, but it honestly made a huge, huge difference for both my mental health and my employability afterwards. I also got to spend some time figuring out what I actually wanted to be doing, so I had a much clearer picture of where my skills and interests aligned after, which helped the search as well.
It probably helps when a FAANG style salary can pay for literal months of rent every single month.
I don’t want to be single again, but the absolute lack of concern over costs was nice.
I've done the same, but with AI generating a lot of the boilerplate, and helping generate automated tests, has accelerated this process considerably.
Unfortunately the leet coding is key to getting the next job
For what it's worth, the two tricks I found for leet-code interviews were 1) do the simple stupid thing first - get something _working_, and then make it good and 2) explain what you're doing and what you're thinking about to your interviewers. I'm still not Great at the leet-code style, but you can get a lot of the way there with some basic knowledge about algorithmic design and complexity and data structures. In general, I've found finishing the problem and being able to talk coherently about it trumps finding the ideal solution.
(I also code like a psychopath - I'm constantly rewriting what I'm writing while I'm writing it, so a) slowing down and b) telling interviewers what I'm doing and why made that look less chaotic.)
I'm also not the person you want to hire if you're looking for deep algorithmic knowledge, though - I've got a long career of delivering high impact systems and solutions, but if you want someone to rewrite the index parsing algorithm in your mature database software to minimize lookup speed, you've got the wrong guy.
I actually just started telling interviewers that I'm happy to discuss code, problem solving, etc at a high level during an interview or walk them through problems I've solved in the past, but I won't do leetcode problems
It hasn't outright disqualified me yet. I also think it's a good "is this team a good fit for me" filter
Toy Software is a lot like working on your bike/car/boat/whatever.
Working on your bike is fun. Working on the bike you need to ride to work tomorrow is stressful.
I miss writing toy software. It’s fun. But inevitably I want to use said software and that’s where the trouble starts. You find all the bugs and none of the time to fix them.
I recently wrote my own invoicing application. I got wrapped up in the joy of adding the features I wanted. Many of these features come at a premium monthly fee for a big named product.
I needed to get an invoice out in a timely fashion. As much as I wanted to use my app, I found certain kinks I needed to work out (with styling, adding addresses, etc.) -- This was where I realized what you have articulated.
At some point, it becomes better to prioritize the "fun" in working on my bike, and the "usefulness" of the daily driver bike.
In doing so, perhaps the fun/usefulness of each converges over time.
Andy Schatz, the independent game creator of Monaco, said something at a GDC talk that really resonated with me along these lines
I'm paraphrasing but -
"Do the hard work that you need to do, but always give yourself an hour a day for the bullshit 'i don't have time for this but I want to do it' stuff.
It'll keep you happy, which will keep you productive through the slog... and those little touches will stand out and make SOME people happy"
For the unfamiliar (the man, not the talk): https://en.wikipedia.org/wiki/Andy_Schatz
I used to self-host my email. I don't any more, for exactly this reason. And of course this applies to every other "just do it yourself!" enthusiast appeal. I actually want to delegate it to someone who does it full time.
Strange example, because self-hosted email has been pretty much setup-and-forget for me. I’ve been hosting my own email for several domains on a $5 VPS for over a decade and it’s largely just hummed along, without needing babysitting. Very rarely, some ISP’s spamblocker gets a little too squirrelly and I have a brief deliverability issue, but those are rare and easy to resolve.
What kind of issues you have with it? I am running my own email server for.about 10 years and very rarely have to look at it.
It was Exim on Debian; generally it would chug along fine for months and then incur some sort of regular sysadmin issue that nonetheless needs dealing with. Upgrades. Occasionally, breaking upgrades. Changes to anti-spam rules on other systems causing deliverability issues. ISP mandating IP migration. Earlier in the process it was an actual, physical rackmount server not a VPS, which incurred extra effort a couple of times to replace it.
And the big one: inbound anti-spam.
It's not that issues are frequent, it's just that there's no guarantee they won't be badly timed. There may come a point in your life where (a) you don't have a lot of free time and (b) you badly need a specific email.
I still have the mail domain, I just pay an ISP to handle it for me.
I heard a lot of people saying hosting your own email server was the best way to be considered as spam by every big email provider. I think it's a big issue, but I've never actually tried it.
I also have more "working software" interests than i have time and energy. Many of them have a ton of mundane boring details i've done a thousand times before. I'm actually a bit excited that i can work on what i want to work on.
With that said it's a challenge and a chore on its own to curate what "AI" writes. So it's definitely not all roses.
Disclaimer: I'm still early in the experimentation phase. We'll see if i hold these opinions in a few months.
I actually only like working on software that has a real use, at least to myself. The coolest is when I spin something up fast at work and people start relying on it.
Same goes for my car, it's satisfying doing inexpensive fixes to keep it running. Wouldn't want some mid-2000s BMW project that's stuck in the garage half the time.
I dunno. Working on a bike you want to use to ride to work could be stressful if it is your only bike, you need it to get to work, and your manager will be really mad if you don’t show up to work one day. If there was any slip available it wouldn’t be so bad.
Actually, I really like this analogy because
1) We can’t really work on non-trivial fixes for our cars because they are too complex
2) We’ve totally structured our living situations around these over-complex beasts
3) Sometimes they do just break down at random, and it becomes a mini-crisis for everyone involved, which can’t really be resolved without missing a day of work to go to a professional
4) The bike is in exactly the same situation, except you can actually fix most bike problems yourself with a little kit
And yet, it is seen as a serious adult thing to have a car, instead of a bicycle.
Some thoughts:
Outside of, like, customer service, most places are throughput-oriented and it doesn’t even really matter all that much if somebody misses a day.
If we really cared about “uptime” of employees, we’d all be living in bicycle range of our jobs, and able to fix our bikes. Maybe there’d be a bus route as well.
As someone who's stuck to owning a single bike for over a decade (tho I will probably change this in the coming year) for both athletic rides and commuting, I'll say the "it is seen as a serious adult thing to have a car" can sometimes make having a single bike worse.
If my bike flats and I'm out of tubes and I have a busy day planned, I have to take it to a local bike shop but those generally close pretty early. If I miss the window to take it to an LBS then I'm SOL. With a car there's generally a developed enough ecosystem that I can get a roadside assistance type service to put a spare on the car until I can actually get my car into a shop.
This is a good analogy for SaaS vs using a tinkerer tool. A tinkerer tool often can be customized exactly to your needs. I can work on my bike at 10 PM in my garage after dinner as long as I have the parts. But sometimes your tinkerer tool is going to leave you working all night or you'll be SOL. But SaaS is generally a full-time job for other folks. Sure you pay money, but there's fewer situations you're SOL.
(Personally when my bike is inop and I need to head to work I use transit.)
You've just identified the next unicorn: AAA for bikes.
> 1) We can’t really work on non-trivial fixes for our cars because they are too complex
Learning how to fix my own cars is something I’ll always be grateful for. They’re not that complex. Most of it is just bolts and a diagram reference.
Sure, if you crack a piston cylinder you just need a new engine, that isn’t really fixable.
Brakes/rotors/calipers, it is almost criminal what you pay a shop for in lieu of doing it yourself. Spark plugs are super easy. Even things like taking the intake manifold off a direct-inject car to replace the fuel injectors isn’t terribly hard.
What has made cars complex is all the electronics that tie into it. The mechanics of an ICE car are mostly about the same.
But,
1) the electronics often are what breaks
2) even if you are good at working on cars, bikes are really, really easy to fix in comparison (I mean, it is a silly comparison).
Of course, they do different things and the thing the car does is much more complex. So it isn’t really a fair comparison. But the fact remains!
> If we really cared about “uptime” of employees, we’d all be living in bicycle range of our jobs
And then reality hits. You're married and your partner's job is on the other side of the city. You just bought and house and now you want to switch jobs but you'd be limited to only jobs in biking distance. eTc....
I bike to work (was 4 miles, now 2), but it's unrealistic to expect people to be in bicycle range of their jobs.
I think reality had already hit my comment, the quote you’ve got there is essentially “if(false)…” haha.
I really just write software for myself only. The time difference (and fun level) between writing software for yourself and writing production quality software is huge.
Feature doesn't work in Safari? Who cares, I don't use Safari anyway. No need to fix.
Perfect example for me is that I use beancount for tracking my finances. It broke one day because I tried upgrading some dependencies from v2 to v3. I tried messing around with the imports and some function names but it didn't help and after wasting 30m I decided to give up on the upgrade.
2 weeks later when I wasn't busy trying to get it to work, I gave it another shot. This time went so much more smoothly and now everything works (fingers crossed since I haven't actually tried out ALL the functionality).
This is why I love building a personal website. It's like a live playground.
>Working on the bike you need to ride to work tomorrow is stressful.
How about working on your bike that other people also work on daily? That's work.
The easy fix is to have two bikes. You can take the time to work on the offline bike, not to mention get needed parts.
I did this for years to learn computer graphics. I spent so many weekends and evenings building weird things that never made me a single dollar, but all the knowledge I gained eventually led to the job of my dreams. Here are some of the weird things I built:
- Tiny ray-tracer: https://github.com/diegomacario/Super-Sunshine - 2D rigid-body simulator: https://github.com/diegomacario/Dyna-Kinematics - Character animation system: https://diegomacario.github.io/Animation-Experiments/public/... - Animation curve visualizer: https://diegomacario.github.io/Animation-Magic/public/index.... - Motion capture playback in the web: https://diegomacario.github.io/Hands-In-The-Web/public/index...
That's a mountain of code that served one purpose: to gain knowledge.
I quite enjoy the spirit of the article, and I believe joy in programming has become even more important in the AI agent coding age we're leaving.
However, am I the only one finding those time estimates way too short?
I'm not the fastest programmer on Earth, but I'm not the slowest either, and I think most of those projects would take me a lot more time than those estimates, specially if I'd be working only 2-3 hours per day.
I feel most of those projects would take significant time researching and learning about the subjects, before even starting to code.
Example: recently I replaced my Pelican blog by my own hacky static site generator written in Odin, working 2-3h per day, it took me 2 weeks -- and this is a simpler project than many on that list.
Your project is vastly more specified than a toy project, because it has a real customer (you) who will expect it to work after it ships. That expectation is what separates toys from real tools.
I bet you could write a word processor in an hour. It might only add or delete one character at a time (no selections). It might persist only to a hard-coded "output.txt" filename. You might have to kill the process because there's no quit operation. But that would suffice for a toy word processor.
Fair point, my example was indeed "shipped to production" and may not compare a throwaway static generator toy project.
I still think those estimates are off, because I think many of those projects would need significant research and learning time, possibly more than actually coding -- not to mention time spent troubleshooting when something goes wrong.
For me, one of the enjoyable aspects of toy projects is that there is no schedule to meet. Which means that I can take my time. And yes, that means I take much more time than the article implies. As an examine, I've been polishing my PEG-based Turing complete language (which features both AOT-to-C and a fully featured repl) since COVID was a daily aspect of life.
I think that if you interpret "X days" as "24*X hours" this list becomes much more realistic.
Who does that though
I see error handling as the biggest culprit here.
When I use dynamically typed languages, it's not necessarily the lack of types that make me write code quicker, it's the fact that I can call anything from anywhere with a complete disregard for whether I should.
When I'm working in statically typed languages, esp. those with explicit error handling (Go, Rust, Haskell, Odin, Zig etc), I find myself spending a lot of time thinking about edge cases, bugs etc because I am forced to do so and find myself thinking in terms of engineering rather than exploring.
I agree the time estimates seem wildly optimistic, or maybe I'm just not cut out for this profession despite being in it for 20 years.
I mean regex is incredibly powerful. 5 days?!?!
I mean, all of those depend on how much was imported as third party dependency, how much was not addressed at all, and what was actually solved manually.
They're all doable if you focus on the core problems. He's got a github with many of the examples up, it's instructive on how to keep things small.
And I think it's important that (based on the code) he had a solid understanding of the problem space before starting to write code. You don't write that tight when you explore a new space.
Edit: https://github.com/ssloy?tab=repositories for the shortcut link :)
I'm surprised by the number of bad takes on LLMs in this thread.
LLMs spoon-feed you with information about how things are implemented. You are not supposed to know how everything works when you start these projects. You're supposed to try your best, inevitably fail, then research the topic and understand where you went wrong, then adjust your approach. If you know how everything works and just follow the tutorial, you won't know what makes other methods fail, and by proxy what makes the one you chose work.
Write a language parser with a regex. Find out that it can't parse recursive statements. You've now learnt that regex can only parse a specific subset of syntaxes. Try to workaround this by pattern-matching the most nested statement first. Find out that it blows up performance. You now know more about time complexity and know what to watch out for when you write a real parser.
Write a non-optimizing compiler from scratch. Find out that you can't make do with unsound optimizations because you can't keep track of what optimizations are applied where. Find out that implementing sound optimizations is hard because you need to track use-def chains. Then you'll understand why SSA is used. Find out that code motion is a mess. Learn about sea of nodes. Merge every optimization pass into one because you're unable to order passes right. Learn how e-graphs solve this.
Write a layout engine. Get stuck on being unable to define what a "width" is. Workaround this with min/max/natural widths, introduce binary search, etc. Learn how this stuff works in practice (this is something I haven't personally done yet).
They say we learn from mistakes. Please don't let the smart (or "smart", depending on how you look at it) machine stop you from making them. It's not a teacher and it doesn't know how to educate.
A lot of people say if you don’t use LLMs then you will fall behind. I’m starting to think that not using them will be a significant advantage in the long run.
I think LLMs improve productivity in the present at a significant cost for the future. It's like cutting an R&D department. You might be able to utilize existing approaches better, but you won't make progress, and I think people are way too overconfident in believing everything important has already been developed.
I guess the counterargument here would be that LLMs could improve research as well by optimizing menial tasks. It's kind of similar to how computing has enabled brute-force proofs in math. But I think the fact that students are still required to prove theorems on paper and that problems with brute-force solutions are still studied analytically should show that tools like computers or LLMs are not at all a replacement for the typical research process.
IMO we are going to see a large class of people who have cognitive deficits brought on by AI tool usage.
I've been wondering lately about how to distinguish between tools that enhance your cognitive ability, and tools that degrade it. Jobs called a computer a "bicycle for the mind," and it seems like LLMs are an easy-chair for the mind. I'm not sure a priori how to distinguish between the two classes of tools though. Maybe there is no other tool like an LLM.
i think theres both. the LLM is an incredible tool you should be able to use well, but its a complement to your other knowledge and tools, not a replacement. if you dont add the LLM to your toolset, youre not going to be building at the same scale as people who are, and if you dont have the backing knowledge, your LLM outputs are gonna be junk because you wont be able to point it in the right direction soon enough in the context window
Using LLMs is like moving to management: you lose your edge on detailed execution, but you improve on accountability and long-term impact.
In the past we used to copy code verbatim from magazines. You have to start somewhere right?
But your brain was the clipboard. That simple process of transcription was something that you couldn't avoid learning from even if you wanted to. You'd notice the connections between the commands you typed and things that happened when you ran the program even if you weren't trying to.
Things would start to click, and then you'd have those moments of curiosity about how the program might behave differently if you adjusted one particular line of code or changed a parameter, and you'd try it, which would usually provoke the next moment of curiosity.
This was how many of us learned how to write code in the first place. Pasting the output from an LLM into your source tree bypasses that process entirely -- it's not the same thing at all.
One thing I've learned about building toy software projects or even just small personal software (stuff that's actually useful) is to avoid implementing flexible configuration engines. I used to fall into this trap where I thought I should make my software to be configuration-centric so that it would be usable for others in case I decided to make my code FOSS. The problem with that is making a configuration engine is a lot less efficient than just writing the code to do the damn thing. I've almost never shared any code outside of small libraries that were very low in complexity, and I wasted a lot of time writing code for other hypothetical people when I should have just been writing it for myself.
> The problem with that is making a configuration engine is a lot less efficient than just writing the code to do the damn thing.
http://mikehadlow.blogspot.com/2012/05/configuration-complex...
It took me a solid decade to accept these lessons. I cannot build a product that even remotely competes with things like Visual Studio and .NET tooling. How many hours would it take a solo developer to recreate a debugger experience that anyone else could be bothered to look at for more than 5 seconds?
The hardest part of building something configurable is the UI/UX. The moment you step outside of the host language, your tooling support goes to zero. Sure, you can pick other popular languages as the language of configuration (SQL, python, Lua, etc.) and piggyback off those ecosystems. But, you still have to build a way for a "non programmer" to visit all of the patchwork sites of "configuration" in a coherent way. Telling the business people to go edit a folder full of .py and .sql files will probably not be received well. Wrapping all of this with a fancy web interface doesn't make the problem less severe either (it makes it substantially worse).
If you have a way to build/deploy your product in <60 seconds from source, worrying about elaborate configuration and customization architecture seems extremely idiotic to me now. The best path I've found is something approximating a common library and a per-customer executable that consumes it and implements the expected contracts. Each customer's project lives in a subfolder on the same monorepo. The power of the hierarchical filesystem is tragically understated throughout our industry.
A rite of passage! :D
[dead]
I've written a ton of throwaway toy applications for personal use.
The tip I can share is: publish them on GitHub.
You'll find out some of them are useful to a lot of people.
I have multiple toy repos with 20+ stars and one with 200+ stars.
Just put one screenshot and simple instructions to run on README.md, no need to write a manual.
As simple as `npm install && npm run start` is enough instruction for most.
The screenshot is important tho.
This is also a great way to stand out among other candidates in the resume-review stage of a hiring process.
The number one question I have as a hiring manager is always "can this person actually write code". A GitHub profile with 2-3 projects with multiple commits to each one answers that question really well. I won't filter out people who don't have that - lots of great engineers have never published code before - but the people who do have that are much more likely to get picked for the phone screen.
The converse is actually a negative signal for me. If you list a GitHub profile on your resume, but that GitHub profile is a ghost town consisting exclusively of "test repo please ignore" and "freshman-year assignments I was required to fork from professor's repo," that tells me you don't have an above-average understanding of why you add something to your resume.
If you don't have an impressive GitHub profile, that's fine. To your point - the absence of it is not a penalty, and the presence of a good one is a very positive signal. But the presence of one that turns out to be a dud is a negative.
It partly depends on the level I'm hiring for. A junior candidate with a GitHub profile full of obvious tutorial projects still helps me skip putting them through FizzBuzz. A senior candidate with a ghost town could reflect poorly on them.
I wouldn't consider a GitHub full of "tutorial apps" as a red flag because it's very easy to underestimate the work behind a project when looking at it from an outsider's perspective.
This is especially true for code that was voluntarily written in someone's free time. For example, each work day is made of 8 hours 5 days a week, but a "hobby day" is more like 3 hours every other weekend. So I would consider one or two features per year to be a pretty productive pace for a volunteer, free time project. Or, worst case scenario, I would just be neutral about it and ignore those projects.
I've been using vhs ( https://github.com/charmbracelet/vhs ) to script gif creation- really showcases CLI workflows if you're demoing multiple commands.
Examples: https://github.com/bbkane
This is great! Asciinema is well-made and a lot of fun, but it sadly doesn't show the recordings in motion right there in the markdown/README. Looks like vhs neatly solves that problem, appreciate the share.
Thanks, I'll try it. I have a few things knocking around.
> Perhaps you’re a user of LLMs. I get it, they’re neat tools. They’re useful for certain kinds of learning. But I might suggest resisting the temptation to use them for projects like this. Knowledge is not supposed to be fed to you on a plate.
I get where this is coming from, and I even agree with it today, but I also want to tag it as "don't cache this opinion too hard". It's interesting to notice when and how our advice for getting help from AI is different from our advice for getting help from other humans. It would be kind of odd, wouldn't it, to put a paragraph at the bottom of a blog post that said "by the way if you have friends who are expert programmers, I don't recommend asking them for help." I think there are two clear reasons that feels odd: 1) expert friends can actually answer your questions and get you personally unstuck, which is huge, and 2) expert friends usually understand why you're doing what you're doing and that they're supposed to help do it yourself instead of just doing it for you.
One thing I bet few people have tried (because I haven't tried it myself) is actually asking an LLM to guide you like an expert friend would, instead of just spitting out code to solve your problem. Maybe they're bad at that, I wouldn't be surprised. But if so, I bet in a year or two they'll be amazing at it. It might be good to build the habit of clarifying what sort of help you need, instead of assuming an LLM will give you the wrong kind of help?
(I'm the post author) Actually, I would recommend that you don't ask an expert friend for help. If you get really stuck then maybe do some light reading about the topic: but the point is to throw lots of things at the wall and puzzle your way through it. Figuring things out from first principles is fun and also provides you with a litany of creative thinking skills to help you tackle other problems. I firmly believe that spending time confused is an essential ingredient in the process.
Obviously, you're welcome to do as you please though, mileage may vary, etc.
> "by the way if you have friends who are expert programmers, I don't recommend asking them for help."
AI is not an expert programmer [today]. and it doesnt take an expert programmer to arrive at that conclusion.
I also bet your expert programmer friend is give you pointers and not writing the whole thing for you
Call it whatever you want, it helps me get unstuck when my dependencies aren't getting along, or when I need to figure out what's wrong with my usage of an API I'm not familiar with, or why my build won't startup based on the code and the vague error message, etc.
I don't care if it's a human or a robot telling me, I just want to get unstuck and I've been getting unstuck more since the advent of LLMs than in the rest of my programming career. Digging through bad documentation and Googling for old form posts of similar-but-not-quite-the-same-issues was always the worst part of programming to me, and I'm glad there's a tool that finally solved that problem.
i am NOT saying they are not useful; they are. especially in domains you are unfamiliar with, or boilerplate, or when being used by an expert in the loop. but that is NOT the same thing as being an expert.
an expert can write 10 lines of code that will be hyper fast while an LLM will write 30 lines that are super slow. both "will work", but one will fall over under load and will be hard to maintain and the other will not.
This is by far my most common usage of LLMs, and they’re good at it. Sometimes you have to be intentional about preventing it from being sycophantic and just telling you that you’re right, through a system prompt or by phrasing the question such that it’s comparing ideas neutrally instead of comparing “your” idea to some other idea.
It feels like Claude and ChatGPT have both become more sycophantic over the past months.
Tell it to ask you questions
I try to do exactly that. I phrase it as using LLM as teachers instead of interns.
I don't trust Dissociated Press on steroids as an intern; why TF do you think I'd trust it as a teacher?!
It's an integrated search engine, why would you trust random people at StackOverflow?
This is quite an impressive list, and many of the things ranking low in difficulty for the author would have been quite high for me. It's definitely inspiring, makes me want to dust off a toy of my own.
That said, I feel the conclusion on learning with LLMs deserves some more nuance. It really depends on how you use them.
This is a terrible prompt for learning:
> Implement this solution for me
This is an AMAZING prompt for learning:
> Please give me an overview of ELF. Focus on the highest level abstractions, more of the 'why' than the 'how'
Certainly one can argue that it takes something away, in that you did not have to conduct the research yourself whenever a question occurred. However, if you are engaging in honest intellectual work (i.e. really thinking), then having a Socratic teacher always ready to discuss your questions is a profound accelerant to the learning process.
I think what you've identified is the "certain kinds of learning" I mention in the post.
If I want to learn about the structure of ELF binaries, I don't figure it out from first principles. I can't: the decisions that went into its design where highly specific to its history and the sensibilities of its creators, and in a modern context a lot of those decisions are near enough random. These are the sorts of things you should use specs, docs, Google, and maybe an LLM for.
What I'm talking about is, I guess, 'constructive' learning. I might not be able to build my own ELF parser through guesswork, but I probably can write a binary format of my own and a simple OS kernel to read it, given enough time. It probably won't look much like ELF, but the process of discovery that'll happen will teach me about why ELF came to be and what problems is solving, and also about the wider design space that sits around it.
My suggestion is that you should not use an LLM's help for the latter.
I like to think of it this way: if you bolted me to a chair in a room with nothing but a laptop, a text editor, and a compiler, what might I be able to create if you left me for 10 years? What couldn't I create because I wouldn't have sufficient implementation-specific information?
Vibe coding via Claude has re-ignited my passion for fun, side programming projects. After a few false starts, I figured out the tooling, processes and in just a few weeks I've built several apps and I'm having new ideas consistently:
-Family calendar/weather dashboard
-Bluesky reader that allows you to hide posts you've "seen"
-Work PM dashboard to make time tracking more gamified and dare I say... fun?
-Chrome Reddit extension that hides posts after viewing for X seconds
-A WordPress plugin to replace a plugin no longer maintained
What I love and had to get used to was looking/using these apps and being happy with them being only 90%. Initially I had Claude make many UI enhancements but I learned to just let most stuff go and focus on the function and building new stuff, not refinement.
I struggle with Claude fixing a bug and not giving me the updated output. Like explicitly having to ask it for the fix 6 times while it insists the update is in the output. Did you ever have that happen?
I read some good advice: give Claude 3 bites at the apple and then burn the session and start over. I find if Claude doesn't get it ready the first or second time the chances of a successful outcome drop considerably. Also "Projects" was a nice low setup way of giving context.
^^ this. Also, not only burn the session - sometimes you need to burn Claude. I'll often find the same problem that Claude struggles with is pretty easy for O3 + GPT 4.1 (or vice versa).
Personally, I just fix the bug myself. It's way less aggravating for me than prompting back and forth many times.
I haven't seen this, unless I don't understand exactly what you mean. I can ask it questions about my code and get answers most easily if I explicitly say "don't write any code".
I have a couple of projects like this and I'm not sure if they'll ever see the light of day. I enjoy writing code, simple as that. Only a small part of my day job is coding, but honestly, it's more like a hobby than anything.
The whole thing with getting people to like your work, using it for clout or padding your resume has an appeal to it that I won't dismiss, but all that takes the fun out of it.
I just don't have much else I actually enjoy in life, so ruining it with peer-pressure, chasing clout, impressing people, worrying about career prospects is a buzz-kill. I used to think that if I am passionate about something like this and I get really good at it, maybe it will help my career and prospects but the real world doesn't work that way. it sucks the joy, fun and passion out of you if you let it.
All that to say it is perfectly fine to like coding for the sake of enjoying it and nothing more. Others play a sport (or watch it), work on arts & crafts, or raise chickens or whatever, I code. I am not particularly skilled or great at coding either, I just enjoy it, and that's it.
If you have fun coding, be protective of your joy! even if you don't take it to the extreme like I do. that's my only point.
I love this take, and I’m reminded of the important idea that while we all are encouraged to be productive, our own joy is also an important thing for us to produce.
I've abandoned tons of toy code but I have 1 thing I wrote in a weekend for myself that I've been mostly blindly using ever since. I just realized while typing this that it's been 8 years. It's a Firefox add-on that I only published to the "store" because running a local-only add-on is a PITA.
It takes regex(s), searches attribute values and/or inner text, and applies a given CSS rule to elements that match. The UI is awful (regex escaped to be inside a json string escaped to be inside a textarea all parsed by javascript) but I know how to use it.
It can probably be used for many things, but I just use it to set visibility: none on elements that link to sites I don't ever want to see (facebook, twitter, linkedin, popular pay/register-walled sites, etc). For example, my HN and reddit views just have blank lines where links any of those sites would normally be. For the most part, I add a site once, and forget they ever existed (except when their CEOs do awful things).
Possibly tangential, but I've really been enjoying writing software with very minimal tech stacks with no dependencies (or very few). Go works well for this, but really any language that lets you create a file and just start writing straightforward code is sufficient. As soon as I need to use some CLI to scaffold out a project with tens or hundreds of files, or pull in a ton of dependencies, I kind of check out. It's fine when you have a team and a big serious project, but in my free time I really just enjoy having a couple files of straightforward code and that's it.
Big time! Almost all of my interesting knowledge came from toy projects I made to solve my own problems (or zero problem solved but it was fun so I did it anyway).
For example, I wanted to know if I can make 1 Kubernetes cluster span multi region, multi cloud. So I slapped TailScale for networking, replaced etcd with multi-region PostgreSQL Aurora, and span the Kubelets between my Raspberry Pi, Digital Ocean VMs, and AWS EC2. And then as the "customer app", I run CockroachDB, rqlite, and tiDB on it (one at a time, I don't want to burn money for this).
It was janky, zero SLA, I likely mis-tuned all the databases, and cost a bit of money :(. But it totally worked, all the db nodes can discover each other. I was satisfied and that's enough.
Another example would be writing my own Raft-backed database, similar to the ToyDB Rust project posted here. Is the DB useful? No. But it is so much fun and I learned a lot.
My AWS S3 bucket and private Git repo are a graveyard full of toy projects. My own Dropbox clone, Pinterest clone, Delicious clone, subset of Google Maps clone, etc. etc. are all RIP in there.
rqlite[1] creator here, happy to answer any questions.
If you're interested in writing Raft-backed databases you might be interested in my talk at GopherCon 2023. It walks through doing exactly that, step by step: https://www.youtube.com/watch?v=8XbxQ1Epi5w
[1] https://rqlite.io
>I wanted to know if I can make 1 Kubernetes cluster span multi region, multi cloud
Can I ask what you had running on that cluster?
As of right now? Nothing important. If I want to use it day-to-day, then I have to care about SLA, then I have to be the sysadmin of that thing.
Not exactly fun :)
What's interesting is that the TailScale network allows the local daemon be exposed on the internet with small settings change. Pretty revolutionary for personal/home IoT projects.
> In 2025, the beauty and craft of writing software is being eroded. AI is threatening to replace us (or, at least, the most joyful aspects of our craft) and software development is being increasingly commodified, measured, packaged, and industrialised. Software development needs more simple joy, and I’ve found that creating toy programs is a great way to remember why I started working with computers again.
Sad but true. I especially feel that comment about losing the most “joyful” part.
> AI is threatening to replace us (or, at least, the most joyful aspects of our craft)
I don’t like this either but every time I use LLMs it feels like we’re talking about completely different things. It moves waaay to fast and makes bad decisions at every turn, if I accepted them all it would be complexity deadlock within a week tops. Pooping out boilerplate sure but then you’re generally holding it wrong anyway (or there’s an opportunity to automate things). Plus even if you don’t have the time to automate it, sure, but then are you enjoying the act of shitting out your own boilerplate?
Out of the things I consider fun the LLM is at best a good rubber duck. It needs constant hand-holding, sometimes polluting the context window (and physical space) with a barrage of poorly written code. Code is bad, we’re trying to minimize it. No? At least that’s how I think: what’s the minimum* amount of code that can solve this problem?
*: minimum in a brain complexity sense, not char count. They correlate strongly though
I find using the LLM as a rubber duck helpful, even if, no, especially if I do not even end up pressing send. Just writing out the problem leads me to the answer.
I have more joy than ever working on toy software while using LLMs. My joy comes from building things, thinking architecture, components, creative solutions that create something new and exciting. Hand-setting lines of code is NOT what brings me joy personally, it's a means to an end. Dancing with agentic LLMs is so much fun to me. I can however understand that if the code part is what brings you joy, then we're probably not moving in a direction that pleases you. For those of us who like to build, first and foremost, it's heaven.
I don't think the hand-setting lines of code is really the part that brings these people joy, the joy is in understanding. LLM-like tools do not create understanding in the user, who is farming out the understanding in order to get results quickly.
I understand the draw and usefulness of this, but just clarifying, it's not the typing that matters, it's what is going on (or not going on) in one's mind.
Ha, these "toys" are quite ambitious. Here are my joyful toys: https://akkartik.itch.io/carousel/devlog
The first example in the article, a regex engine, clicked for me. I had read the book on beautiful code #, and the first short story was about a regex thing by none other than Dennis Ritchie. As Fate would have it, my job required me to make a tool for web scraping. Inspired by that book I created a pattern matching language for dom trees. And a click and point tool to create those patterns. At the best time or company had a full time person maintaining maybe 10k patterns to scrape days from library catalogs... It was not a toy project, but it did feel like, even if I got paid to do it.
# Andy Oran, Greg Wilson: "Beautiful code"
Took a minute to track that down (Oran is spelled Oram?) https://www.oreilly.com/library/view/beautiful-code/97805965...
This is one thing I don't get about the LLM people. For most of us who are professionals, writing software is not very hard, and it is in fact fun!
I don't need to let a computer write my software. I already know how, and I enjoy it. I need the computer to do the hard part (gathering requirements, speaking with stakeholders, etc).
Software is a joy.
There are several sources of joy in programming. Some lean more toward the process and exploration of the problem domain, others more toward the outcome and impact.
I had a shift of perspective at some point where I realized that software wasn't just fun to create, but you can solve problems with it, including your own, which is incredibly rewarding and kinda feels like wizardry.
Ultimately I find both to be enjoyable, though in terms of career and life outcomes, focusing on solving real problems and producing tangible outcomes has had more of an impact than noodling with neat implementations that nobody but me cared about.
Turns out if you solve your own problems (or just publicly work toward solving them), there are often other people who also struggle with the sam problems, who may throw money your way ;-)
I've done a lot of these kinds of projects. I got really sucked into a rabbit hole with the parser/interpreter/compiler that I was stuck (happily) in "I need to understand this so I will build one" mode. I eventually built a database server and query language for it. (for a business) This was in 2009, but I highly identify with his method of learning by doing for fun!
What's sad is how difficult it is to write software today. In the old days your dad could buy a C64 and cobble together an application. It should be vastly easier to do the same kind of thing with vastly better building blocks today. Why can't some Grandma drag and drop some widgets and have a recipe manager with sharing features amongst her friends and family?
Here's a few disorganized thoughts in good faith
1. Because half her friends and family are on iOS, and that means fighting the App Store. (This is a social problem essentially, in fighting Apple)
2. Because networking is hard. How would you have shared recipes with a computer in the C64 days? Email? BBS? (There are partial technical solutions to this, but they would require people to run something like friend-to-friend overlay networks)
3. Because most stuff happens in web browsers and that means pay-to-play, or vendor lock-in, or using AWS free tier and being a programmer. (Ass, grass, or cash, nobody hosts for free. Friend-to-friend networks may also help with this)
4. Because a recipe manager with sharing is best implemented as just emailing your recipes to your friends and storing them as txt files locally. Anything more complicated is beyond the scope of a Visual Basic-style drag-and-drop WYSIWYG anyway
5. When was drag-and-drop enough? The widgets need code behind them to save and open files, right?
6. You might be kinda onto something, and the longer I write async code I more I think the programming world is ready for another big pruning. Like when structured programming said "No goto, no long jumps, if-else and exceptions is all you need", we might be ready for "A single control thread which never blocks and never awaits, and delegates to worker tasks, is all you need until you are building some C100k shit"
When I was a teenager, I imagined that after a decade or two of working with computers, I would be able to write a computer game over a weekend. Or maybe two weekends. I had a notebook full of ideas and sketches, so that when I am ready, I will make all those amazing games.
I even made a few (quite simple, from my today's perspective) games in Pascal during high school and university. I expected to become much more productive over years of practice.
That didn't happen, for several reasons.
First, my expectations for a good game have increased. I don't want to make the most complicated game possible; I am perfectly okay with 2D raster graphics and simple algorithms. But I expect a good game to have animations, sound effects, at least ten levels that feel different, and an option to save game progress. My old games barely had half of that (some were animated, some had ten or more levels, only one had both of that).
Second, things became more complicated. It is no longer "320 x 200 pixels, 256 colors". Windows are resizable; different screens have different sizes. Programs need to be aware that multiple threads exist. Sometimes there are many possible choices, and I get paralyzed by choosing between them. Programs are expect to have installers; it is no longer enough to have one EXE file, and optionally a few data files together in a ZIP file. It felt like every time I mastered something, a new problem appeared to be solved.
Third, as a teenager I didn't realize how much my everyday work would differ from the kind of work necessary to make a computer game. Some skills are transferable: I am more comfortable with using threads, parsing data files, writing algorithms, the program architecture in general. But many skills are not: if my dream is to make a desktop application, then e.g. all the web frameworks that I have learned over those years are useless for this purpose; and they have cost me a lot of time and effort. So from the perspective of making computer games, as an adult I maybe learn in five years as many relevant things as I have learned as a teenager in one year, when I had lots of free time that I could dedicate to this goal.
Fourth, life gets in the way. There is much less free time, and much more things that I need or want to do during that free time.
So here I am, after a few decades of IT jobs, and (a) I can't really make a complete computer game over a weekend, and (b) it's irrelevant, because until my kids grow up I probably won't get a free weekend anyway. Or rather, even the rare "free" weekend (when the kids are away) is spent on other things that have higher priority.
How hard did you look? WordPress has a few recipe maker plugins if you didn't want to code anything. Just install one and password protect the whole thing, and then teach (and write instructions for) Grandma to use it.
In the age of powerful computers, you can use Hypercard on an emulated Mac, you can use any number of hypercard-clones out there. She can just use Google slides. etc.
I think the main difficulty is deployment. Grandma wants that recipe manager to be available to her family 24x7. How can she deploy it easily for free or very low cost? If there were a modern Hypercard, I think the key to its success would be making deployment extremely simple, reliable, and safe.
NextJS with Neon on Vercel has a capable free tier, and there's enough training data that LLMs are decent at it. If Grandma is that interested in building an app, I'm sure she'd love to spend a few hours with a grandkid to set things up and then being taught how to vibe code (and also how to call said grand kid for help).
There are platforms like Observable and Repl.It that just let you deploy code/data pretty quickly.
> Observable
> Repl.it
Sorry, Grandma has never heard of those fancy-pansy platforms.
How about a very reliable device with an interface that even a child can understand, deployment process that consists of only one action, and no upkeep costs? Like, for example, a notebook... Not everything has to be computerized or software.
> Why can't some Grandma drag and drop some widgets and have a recipe manager with sharing features amongst her friends and family?
Because there's no money in it.
But regardless of the time estimates, I appreciate the idea!
I was going to make a similar comment. This list is fine, but the times are all over the place. If they removed those estimates it'd be a much better list. If a hash map takes 3-5 days for someone, they aren't making a threaded VM in 7 days.
This person is a game developer who works on an open source voxel RPG https://github.com/veloren/veloren - take their estimates with that in mind!
I have to assume the author listed these times based on how long it would take for someone to follow along with the blogs that they linked to, which demonstrate stripped down simple versions of these things. Like the POSIX shell example just implements a few basic things, but leaves out a lot (and says so).
This is the point, yes! Toy software is not production-worthy. It might not even be sufficient for your own personal use. It's about building proof of concepts and rapidly scouting out the terrain of a field you don't know much about, and getting a souvenir to remember the experience with.
Oftentimes I find that I enjoy one of the projects so much that I take it further than 'toy', but that usually ups the time estimate by an order of magnitude.
I really like the idea purposed here but none of these projects are even remotely interesting to me. It's moments like this when I start to question whether or not I'm losing my love of programming.
Perhaps. I scanned the list and most of them sound like interesting projects that I would like to tackle at some point (or that I have tackled in the past for "fun").
Honestly, I think I was in your same position until recently, then I had some time off work thanks to the newborn and decided to start a "simple" coding project. Since it was purely for fun and to challenge myself, I decided to try and avoid bringing in any dependencies and wrote everything from scratch. This made it much larger and more complex than I had initially anticipated, but despite that (or, maybe, because of that?) I longed for those few hours when I could tinker on my project. Coding suddenly became fun again.
Maybe you're just a bit burnt out?
Definitely a possibility
>An ECS framework, 1-2 weeks
That gave me a good chuckle. Yeah look, you can change your data layout to be in more of an ECS format, make some optimizations and call it a day; but in reality you're still going to be pretty far from the real thing.
I can say this because I've been working on one for some months now, of course I'm going for a production build and also to turn it into a Unity asset that's worthy of being sold, so it's going to require a lot more effort. The goal of mine is to ensure that its extremely simple to use (which Units DOTS totally fails at) while still being as efficient as possible.
Some notable things I've had to do for this project:
- I have a tool releasing on the Unity asset store (currently being vetted) that performs optimization for you, e.g. you give it a variable such as an int, a range of let's say 1..1024; then it'll automatically time your code while searching for the best sample in the range. I needed to create this for the next item, which is
- My C# Thread tool which runs managed threaded code for you. It creates no garbage so and uses the above optimizer on three different variables so as to find the best amount of threads, calls and inner loop sizes to use for your code. Because of that it can run managed code almost as fast as Burst/Jobs (whereas they use their own compiler and optimize code instead of using Mono). It can even beat Burst on small to medium workloads as long as the code inside doesn't contain some efficiency that Burst can natively fix.
- Then of course there are other things an ECS needs like pathfinding, unit collision avoidance etc. For these I have both CPU and GPU solutions since CPU only really works up to about 10k entities at a decent framerate.
So yeah, I don't recommend trying it; it's probably a total waste of time.
Have you seen where he’s using it though? His GBA Mario 64 port is one of the most impressive homebrew projects I’ve seen in years.
Oh yeah make no mistake, he's super talented!
I added up a score for myself from that list giving me one point for something that more-or-less matches an item from this list and a half point for something vaguely similar. I got 15.5
The compiler, chess engine, and text-editor have been targets for years but I haven't quite gotten around to doing them.
I like the idea of the bitwise challenge.
I had been thinking that another challenge that I'd like to see (and have a go at) might be a plotPixel Challenge.
Make a game that does all rendering with
Unsure on byte for 256 indexed color or 32 bit for setting 24 bit color from ARGB as r= rA+R, g=gA+G, b=b*A+B.It would be easy enough to test if a game obeyed the rules because a full video capture of the game should be possible by intercepting just those functions.
Perhaps a (generous) source code size constraint to stop people going too crazy. (maybe 64/128/256k source-code, zipped). Not for code golfing as much as to avoid people trying to squeeze Crysis though a straw.
I'm writing a shell right now in ruby after seeing a few other folks doing so and looking at the current state of shell development and not finding anything I really liked. It is indeed, a lot of fun. Using Vidar's really quite nice text editor to do so is equally pleasurable.
https://github.com/vidarh/re
Got a few PRs for you when you get to this, friend.
I agree. It’s fun to just write code to solve problems or even for fun. I love go (it’s the best all around language and it’s not even close) so for fun I wrote a game engine and a small game that I might try to turn into an actual game. It’s good to keep fresh. I’ve also tried to do llm free days
> Orbital Mechanics Sim
Here's a little toy I made. Try to keep the comet on-screen by clicking to spawn a planet: https://ssl.pepas.com/gravity/gravity.html
Written in C / SDL 1.2: https://ssl.pepas.com/gravity/
I agree with the sentiment in theory, but the time pressures of adult life make this difficult.
> What I cannot create, I do not understand
I suspect Mr Feynman would have phrased this a bit differently if he had first-hand experience with the modern front-end technology treadmill.
I love the article but disagree with the final point. It wasn't until LLMs came out that my toy projects have all been greatly upgraded with features and functionality. I simply didn't have time to do so before.
The point is not the final product, the point is the process.
As a toy project I can recommend re-creating the core functionality of Git. Its basic functionality beautifully simple.
Checking out Linus’ first commit of Git and comparing it with my own solution was also very interesting (and humbling).
If anyone is interested, here’s my toy Git implementation in Go https://github.com/emanueldonalds/shit
I know writing "toy" software has exposed me to new tools I might not have otherwise even known about.
I can't afford much n the way of software, so I've been trying to learn to make do with FOSS.
Problem/Truth is that writing toy software is not profitable - that's not to say one must always work on profitable things; what i mean is that it's being kept unprofitable, because in a hyper-saturated monopolized attention economy, every piece of "toy" software that isn't a video game or a plugin to some proprietary app ecosystem is essentially an act of subversion, an attempted "theft" or piracy of what the Company considers its rightful property: the sum of all disposable human attention.
That sounds as good a reason as any to become a pirate, frankly
I love that attitude! This is essentially my preferred answer to "what would people do in a post-AGI world when they don't need to work any more".
My kids (7-10yo) are currently learning programming. I’m debating if I should show them the LLM. Leaning towards no :)
I share the same feeling. Programming can be an entertaining and joyful activity where enthusiasts don't really need to worry about creating something useful that will be shared with others.
It's like cooking a beautiful and tasty gourmet recipe that serves only one.
gustavo!!! your website was the first thing i thought of as i was reading this blog post. im a huge fan of your teaching style. thank you for everything you do! :-]
Very nice collection of toy projects, I resonate with you, I also wrote a ton of toy projects in sabbaticals, check the better ones out here : https://milgra.com/
A one-sentence description would be a great addition to these projects. "SOV|APP|VIDEO|SOURCE" doesn't really invite a click.
Ever since Nintendo let the mask slip and got up to their old evil bullshit with the Switch 2 launch, I decided to dust off the solution my teenage self came up with: I'll just play the video games I write. Even if they're small and amateurish, they're worlds and experiences crafted personally for me. And I don't have to pay a cent for them, let alone put up with "key carts", DRM bullshit, or we-can-remotely-brick-your-console bullshit.
I'm sure I'll keep buying commercial games from companies with reasonable terms (GOG, maybe?). But writing my own is great fun. It'd be nice if someone else enjoys them, but they're mainly for my own enjoyment.
Oh, and I "vibe code" the old fashioned way: with a REPL and some chill tunes.
What sort of evil bullshit did Nintendo get up to with the Switch 2 launch? I don't follow gaming news.
1) High prices for the console and games, which may be partially explained by inflation, but still, shit's steep. We're back to the early 90s days of scouring pawnshops for deals and carefully considering which few games we really want to buy. Plus, $10 for a sales demo (Nintendo Switch 2 Welcome Tour).
2) Console must be registered online before becoming playable in handheld mode.
3) Nintendo recently released a terms of service update in which they reserve the right to disable your console if they catch you violating acceptable use. Switch 2s found using piracy carts have not yet been bricked this way, only locked out of internet play (like original Switches were), but still...
4) Some Switch 2 games will not get a physical release, only a cartridge which provides an "activation key" for a digitally downloaded game. Why even bother with this bullshit, I don't know, because it's all of the downsides of physical media and all of the downsides of digital downloads, with none of the upsides of either. On a console.
I don't really get a lot of joy out of building copies of things other people have done better than me before. Almost by definition, no list of cool toy projects is going to appeal to me for that reason. I might like the techniques or algorithms employed within them, but frankly if I'm after that kind of knowledge I just find Leetcode and traditional studying to be both higher value and more fun.
But I do like writing programs in some strange new, non-CS domain where very little software to address my very particular needs already exists. Most of the software I've written in the last few years has been of this kind, for the purposes of learning Finnish. It is amazing what a few shell scripts, fzf wrappers, and well-timed calls to an LLM can do for accelerating one's own learning process.
An anecdote about how a toy project _really_ helped me out:
I work on Dart professionally. I've also been tinkering on a toy programming language that may never see the light of day. My toy language has gone through several incarnations but a while back I was working on adding algebraic datatypes, pattern matching, and exhaustiveness checking to it.
The canonical algorithm for exhaustiveness checking is this paper by Luc Maranget: http://moscova.inria.fr/~maranget/papers/warn/warn.pdf
When I first started dabbling in programming languages over a decade ago, I tried to understand that paper for weeks and just could not wrap my head around it. I don't have much of a formal CS background and that paper was impenetrable to me.
So here I am tinkering on my toy programming language and I run into again. I give it another try and laboriously implement it in my interpreter, basically doing a straight translation. I don't understand the code, but it seems to sort of work. So then I start refactoring it a tiny bit at a time into a style that fits the way I think about code. Eventually the algorithm sort of folds into itself and before I know it, I finally understand how it works. The end result was only a page or so of code, but my toy language had real exhaustiveness checking work.
Meanwhile, at work, I am working on adding pattern matching and exhaustiveness checking to Dart [1]. Exhaustiveness checking here is a much harder proposition because Dart has subtyping, unlike my toy language and the ML languages that Maranget's paper works on.
I'd been hacking away at an exhaustiveness algorithm for Dart based on a couple of papers about Scala's approach but they were sort of approximative and inelegant (in my opinion, maybe they are a great fit for Scala).
But once I understood Maranget's algorithm from implementing it in my toy project, it finally clicked for me how it could be adapted to work with subtyping in a sound, coherent way. I wrote it up as quickly as I could and (with a lot of additional help from a teammate to handle generics), that became the algorithm we shipped:
https://github.com/dart-lang/language/blob/main/accepted/3.0...
It wouldn't have happened if I hadn't coincidentally been working on a toy pattern matching implementation at home.
[1]: https://dart.dev/language/patterns
I love this, this is exactly the sort of thing I've experienced too: building the toy project gave me an on-ramp to something similar but much more complicated in the future.
Dupe: https://news.ycombinator.com/item?id=44284291 (117 points | 8 days ago | 28 comments)
Seems like the link changed, and thus not caught by the dupe detector.
Apologies, I overhauled the entire website (backend and all). I wrote a post about it, if you're curious.
These time-to-complete estimates seem reallyyyyyy aggressive to me. I dunno maybe I'm just the slowest programmer in the world.
yeah, it's a good post overall, but the humblebrag factor undermines it
Very good article, but am I the only one who is seeing that this guy got Mario 64 working (somewhat) on a GameBoy Advance? That's insane.
Agree and that's why I create https://github.com/ai-glimpse
Everyone is caught up in focusing on LLMs instead of thinking about why smaller software is dramatically easier to write.
Software is like baseball. Never stop practicing.
Consider it like exercise. Pro atheletes train and exercise. Pro software developers should train and exercise their craft.
I'm always mixed on this. In the moment it can be fun. In the end when it inevitably feels useless, idk it feels bad. Like I spent a ton of time on something and get some knowledge out of it. It doesn't make my job better and I guess doesn't improve my situation much. My rust sub-directory feels like a digital closet of rpis collecting dust.
The things I find fun writing and learning about and the tools I use to do it are so disconnected from anything i do professionally.
>Perhaps you’re a user of LLMs. [...] But I might suggest resisting the temptation to use them for projects like this.
I agree. However: Your best bet is likely to build it by hand and then see if you can get an LLM to build it.
It is looking like we are living in a world where an important tool to have in your toolbox is going to be the skills and experience of working with the LLM/AI tooling. There are skills, knowledge, tools, tips and techniques that you need to develop to effectively use these tools, just like the honing you get from writing the code.
Over just the last half hour I did a brain dump of "hot tips" to a friend of mine who is just starting to use Claude Code to kick-start him.
The LLMs are REALLY good at writing small toy programs, and at the end of the day it's great to have little tools that you'll use every day.
Everything is “a joy” these days, much less “could you elaborate”.
Good list of projects. I have to disagree on the point on usage of LLMs.
These projects are going to be sideprojects and with busy schedule it might get tough to find motivation. Using LLMs for some of the parts can give the momentum required to continue.
We can also learn from studying the output generated by LLMs.
Absolutely love this article, couldn't agree with this more.
it's more of compression. fastest way to get 10 yrs of backend sense in like 3 weekends is build fake OS scheduler or just kv store with fs persistence. and the shortcuts your brain takes after that, scary fast. you just start seeing stuff everywhere. like oh this LRU cache is just dumb DLL with hashmap. or this parser is just recursive descent with some peek-ahead logic stapled on. and then you can't unsee it. you start breaking down every abstraction you touch. even prod libs look like toys.
also lol toy projects expose your defaults hard. how you name vars when there's no PR. how you layout dirs when no one's looking. do you even test stuff or just printf until it works. do you log or do you trace or just guess.
love the spirit of the article, the time frames listed are quite unrealistic however
I’d triple the estimates, really. For example a game boy emulator is not trivial; there are far too many edge cases with a zilog. Chip8 would better fit for that time frame.
It seems to be an unpopular opinion here. But the last thing I want to do when I get off of work is touch a computer. I haven’t desired to be a hobbyist developer nor have I written a single line of code that I haven’t gotten paid for since the day I walked across the stage with my CS degree in 1996. I was hobbyist for 10 years before then.
Life is too short to spend in front of computer.
Writing software funds my joy.
If you're allowed! Apple says no
[dead]
Overengineering is a joy. Using LLMs is a joy. Building something novel, bespoke, something you wouldn’t do when ruthlessly prioritizing for profit.
Waste of time, you wont impress anyone because there are no more software jobs left.
Go to gym.
Who am I impressing at the gym? The weights have already been lifted
So you don't look like sh* that's more impressive than cranking out code that absolutely no one cares about. Especially things that have been done 100s of times.
Please work out. But also please write software if you want to improve your skills.
udk what you're missing out if you aren't in the gym twice a week.
Reinventing existing software has 0 merit. If you cannot be creative enough to invent new things, you shouldn't bother working.
That's rubbish. Reinventing existing software as a learning exercise has plenty of merit.
No it doesnt. Build new things or dont waste your time.
Learning exercise lol.
I don't believe you understand the word iteration, why that word exists, or the benefits it provides to society in general.
How are you supposed to learn to invent new things if you don't know how existing things work?
There's ZERO room for innovation in the projects suggested.
Wanna do some creative thinking? Read some books, learn how normal people think. Explore latest datasets. That's where the meat is. 98% of what you read online has been written by super uncreative(boring) people. They have no imagination whatsoever, IQ is well statistically speaking not too flattering.
Internet is ran by the bottom. It's slop by the bottom, for the bottom. So you need to hang out with creative people.
Tech people write the most boring blog posts because it's aimed at getting hired. Nothing of substance is shared. That's why gwern stands out like a sore thumb. The majority of human beings are incapable of creative thought.
> There's ZERO room for innovation in the projects suggested.
I do not agree. Improved variants and innovation would be possible with operating systems, GUI, programming languages, etc. It would also be possible that something will be made better for one purpose because others are not as good for some purposes, but it is not the only way to do, though.
You learn how to do these things, then innove in other projects. I don't see these two concepts contradictory.
Are you ok
Novelty is not the only value. Slop comes from your attitude: people like you who think that things must be new to be worth something
That philosophy has some lingering halitosis of truth to it.
Any inspection of any economy suggests otherwise. How many tire manufacturers are out there? How many different brands, styles, and types of tire? How many total US Dollars per year are those companies sharing?
How about oil changes? Car washes? How about car wax?
New doesn't equal profitable. Making a different product, or service, is where most of the profits on earth sit.
Wanna be a rebel and broke? Then by all means follow the new-new and barely break-break even.
I guess you think you're the first person to express that sentiment?