It might just be a me problem, but I've always been wary of regexes. They're not too bad to write, but reading them back and understanding what's actually going on can get a bit hairy. Plus, all of the subtle differences between regex libraries seems like a bit of a footgun.
Obviously they have their place, but I know a lot of the older guys seemed to love them way more than the young.
Various libraries (e.g. Python's `re` library) support comments and whitespace as an option allowing you to format the regex on multiple lines with commenting to document what each part does.
I'm not sure if there are any regex libraries that support DSLs and easy composability (e.g. the email RFC regex would be easier to read/maintain if you could specify the individual parts like are defined in the RFCs).
Something that seems obvious but not always implied by people's comments is that people are rarely trying to match an entire document with a regular expression so it doesn't really matter that "HTML is not a regular language".
If I am trying to e.g. count div tags with a regex like "<div" or whatever, then clearly this would work in 99.9% of cases and probably achieve what the poster is looking for.
As soon as you also add character classes to ignore various parts of the document that you are not interested in like "<div[^>]*>" or whatever it is, then it is eminently useful even if the bit we are ignoring is not fully regular.
One lovely thing about regex is how fast it is. I was asked to parse a massive CAN Bus log file for how many times some event had logged. This was the early 2000s and the file was 6GB, which was pretty big. I tried .Net's string.StartsWith or something and that took ages to run through the file. I did the same thing with a regex and it finished in like 5 seconds (HDD, not SSD!). I don't know how the magic works but it is very impressive.
Are we also counting divs inside comments or literals within scripts?
There is a reason this advice is default. The chances an edge case exist are probably a lot higher than anyone is prepared to accept. Even in the "simple" cases.
Regexes are great, they seem like magic when you use them right. They can solve your problems even if you don't use them right. Just make sure not to mix the flavors.
Before the AI craze, I'd gotten quite good at writing regexes. Regexr was quite useful for decoding and composing them. I feel like they're going to become a lost art.
Totally agree. Selfishly, I was always the "regular expression" guy because they were a bit hobby space of mine (engine implementation and such), so seeing LLMs rip them is a bid of a bummer.
Half the reason it's a bummer is because I've seen coworkers who don't know when a regular expression is very suboptimal performance wise, but the LLM has no problem spitting it out. Part of really understanding regular expressions is knowing when to not use them.
The one that sticks in my head is when I was debugging some code that I was suspicious was causing our high memory consumption on a simple API service just to find out the regular expression was being used to strip a potential "data" front of a base64 encoded file (apparently someone thought we should do that instead of rejecting the payload). The regular expression scanned an entire base64 string that was up to 50 MB for the raw file, so about 66MB base64 encoded. I'll tell you what, replacing it with a loop over the first handful of characters solved all the problems. It should've never been a regular expression. If you see regular expressions as an archaic language that solve string problems, and now the magic box can make them for you, you're in for hell.
Just like jq there will be some lad along to tell us that "I don't like the syntax and find it confusing" not realising that's the exact superpower it presents is it's terseness is a key property to it's adoption. jq and regex really are sort of handy one liners that you invoke in other scripts and you explain what they do in your script with a comment.
OT but this me-problem makes me angry every time I read it. Nothing is simple, otherwise it is trivial and not worth mentioning. I can't read over this without thinking that I'm not smart enough to wrap my head around something instantly.
It might just be a me problem, but I've always been wary of regexes. They're not too bad to write, but reading them back and understanding what's actually going on can get a bit hairy. Plus, all of the subtle differences between regex libraries seems like a bit of a footgun.
Obviously they have their place, but I know a lot of the older guys seemed to love them way more than the young.
Various libraries (e.g. Python's `re` library) support comments and whitespace as an option allowing you to format the regex on multiple lines with commenting to document what each part does.
I'm not sure if there are any regex libraries that support DSLs and easy composability (e.g. the email RFC regex would be easier to read/maintain if you could specify the individual parts like are defined in the RFCs).
I honestly never knew that, should give it another go.
Something that seems obvious but not always implied by people's comments is that people are rarely trying to match an entire document with a regular expression so it doesn't really matter that "HTML is not a regular language".
If I am trying to e.g. count div tags with a regex like "<div" or whatever, then clearly this would work in 99.9% of cases and probably achieve what the poster is looking for.
As soon as you also add character classes to ignore various parts of the document that you are not interested in like "<div[^>]*>" or whatever it is, then it is eminently useful even if the bit we are ignoring is not fully regular.
One lovely thing about regex is how fast it is. I was asked to parse a massive CAN Bus log file for how many times some event had logged. This was the early 2000s and the file was 6GB, which was pretty big. I tried .Net's string.StartsWith or something and that took ages to run through the file. I did the same thing with a regex and it finished in like 5 seconds (HDD, not SSD!). I don't know how the magic works but it is very impressive.
Are we also counting divs inside comments or literals within scripts?
There is a reason this advice is default. The chances an edge case exist are probably a lot higher than anyone is prepared to accept. Even in the "simple" cases.
Regex can also be horribly slow - it depends on the particular regex you are using.
Regexes are great, they seem like magic when you use them right. They can solve your problems even if you don't use them right. Just make sure not to mix the flavors.
Before the AI craze, I'd gotten quite good at writing regexes. Regexr was quite useful for decoding and composing them. I feel like they're going to become a lost art.
Totally agree. Selfishly, I was always the "regular expression" guy because they were a bit hobby space of mine (engine implementation and such), so seeing LLMs rip them is a bid of a bummer.
Half the reason it's a bummer is because I've seen coworkers who don't know when a regular expression is very suboptimal performance wise, but the LLM has no problem spitting it out. Part of really understanding regular expressions is knowing when to not use them.
The one that sticks in my head is when I was debugging some code that I was suspicious was causing our high memory consumption on a simple API service just to find out the regular expression was being used to strip a potential "data" front of a base64 encoded file (apparently someone thought we should do that instead of rejecting the payload). The regular expression scanned an entire base64 string that was up to 50 MB for the raw file, so about 66MB base64 encoded. I'll tell you what, replacing it with a loop over the first handful of characters solved all the problems. It should've never been a regular expression. If you see regular expressions as an archaic language that solve string problems, and now the magic box can make them for you, you're in for hell.
Some people, when confronted with a problem, think "I know, I'll get my agent to solve it with regular expressions."
Now they have three problems.
Just like jq there will be some lad along to tell us that "I don't like the syntax and find it confusing" not realising that's the exact superpower it presents is it's terseness is a key property to it's adoption. jq and regex really are sort of handy one liners that you invoke in other scripts and you explain what they do in your script with a comment.
"Doom Using Regular Expressions" https://news.ycombinator.com/item?id=49094081
The last thing we need is huge regexs made by AI
Been burned by this exact thing before
> The regular expression is very simple
OT but this me-problem makes me angry every time I read it. Nothing is simple, otherwise it is trivial and not worth mentioning. I can't read over this without thinking that I'm not smart enough to wrap my head around something instantly.
Every time I cut-n-paste a regex into code, I comment with the url of the spell book page I copied so future me can answer, "WTF does this do again?"
I'm wary of external urls in code. Some plaintext comment would come in handy for the day the link inevitably goes dead.