>The Only Programming Language Built on Mathematics, Not Fashion
As a modern array language D4M is the natural successor for SQL [1].
D4M is based on mathematics like SQL, specifically associative array algebra but not relational unlike SQL. It's more generic since can it caters to most modern data abstractions including spreadsheets, database tables, matrices, and graphs [2].
You can achieve 100M database inserts per second with D4M and Accumulo more than a decade ago back in 2014 [3].
[1] D4M: Dynamic Distributed Dimensional Data Model:
I've been slowly transitioning from using an ORM to just plain SQL. It's so much simpler. Less magic, more explicitness, and more control. Also, much better performance. I think the thing is to construct your model around the different queries you need to perform. In many cases, especially a CRUD-type situation, you'll end up with 10-20 different SQL queries, and that's it.
Once you break free of ORM’s I find the code so much simpler to maintain.
Here’s the query(typically multiple different subqueries and return types), here’s the params, give me all the data back and something like Dapper in .net is an absolute godsend to convert it.
Helps simplify complex SQL queries and no need to waste network traffic on data that client side is never going to use, and waste CPU cycles processing it.
Yes, what about database portability?
I am on my 50s and it only mattered on a single project, which was anyway a middleware for application servers.
For sure, but have a solid grounding in set theory to go with it.
I've dealt with so many poorly-performing stored procedures that ended up being written as iteration over a CURSOR when they could have been done with sets. Programmers who don't grok set theory reach for iterative constructs which, while they work fine, are an impedance mismatch with SQL.
- I recently read that most programmers SQL knowledge is outdated by 20 years and it’s true for me. There are quite a lot of features in most DBs that feel very "new" to me.
- Comparing SQL to React weakens the argument. SQL is the language, React is a piece of software. You certainly can run 30 year old JS today in modern browsers.
For me SQL has long been the gateway to the world of development. I work in the UK non-profit sector and traditionally this kind of technical knowledge is rare, so for any team I've worked with I've built learning pathways that start with SQL before pushing out into Python, Linux, and other things.
We're not exactly at the bleeding edge of current technologies, but SQL has consistently proved to be a great jumping-off point for novices who have even a passing interest in computing.
> Edgar Codd formalised relational algebra in 1970. SQL sits on top of it as a declarative interface. You describe what you want. The database engine decides how to get it. The engine improves every year. Your query stays the same.
Although SQL is of course not relational Algebra (and others like Datalog and D4M are better), it's still cool. It inspired kSQL like Lil uses https://beyondloom.com/decker/lil.html#lilthequerylanguage , which inspired the code I'm most proud of: https://codeberg.org/veqq/declarative-dsls A common query language, a common idiom, for many data structures (arrays, hashmaps, datafremas) is liberating, permitting you to e.g. solve sudoku, make mandelbrot sets or calculate primes directly:
(def n 40) # to reach primes up to, left is sqr of n, right n/2, then multiply them for rows
(def composites
(df/select :from (range 2 (+ 1 (math/floor (math/sqrt n))))
:cross (range 2 (+ 1 (/ n 2)))
:where |(<= (* ($ :value_left) ($ :value_right)) n)
[[:value_left :value_right] :value
|(* ($ :value_left) ($ :value_right))]))
(df/select :from (range 2 (+ 1 n)) :exclude composites)
I've learned SQL around 20 years ago, and in all this time I've felt it was just a poorly designed language. It was always infuriating to write because of its verbose nature. Keywords were split into two words. I'm still shocked it's not "GROUPBY". There is no composition and modularization of logic, queries become massive expressions.
I know I'm in the minority in places like this, but I've spent all my life using ORMs, and never once regretted it. And I'm the kind of person that actually likes low-level C from time to time. SQL just feels like a poor abstraction layer: either go higher or lower.
It’s a good abstraction layer, and a fundamentally good/effecient model of organization and data management. It’s a horrible language, has a meaningless standards doc, some of the worst debugging tooling of modern system and generally any tooling outside of the RDBMS engine itself is 20 years stale.
The only difficult part in arguing this is that RDBMS != SQL != RelationalAlgebra, and it’s very often forgotten
Same can be said for learning an OO programming language or a procedural programming language. I learned C++ at school and started using Java on my first job. I forgot how to work correctly with pointers but I have tried multiple languages (using the same paradigms) and managed to build working software
Everyone knows SQL already. The harder parts that pay off are schema design, knowing how to interact with your DB in code, and knowing all the ins and outs of whatever DBMS you're using.
I would emphasize the importance of batching and set operations. This is where I think many developers lose track of the rabbit, because you don't have much control over either of these things via ORMs. You have to get your hands dirty with raw command text.
The value of this stuff is difficult to overstate. Batching allows for you to rapidly load the RDBMS. The first few times you test, it will probably go so fast you won't believe it loaded anything at all. Set operations allow for you to bring this newly loaded data to visibility in production tables nearly instantly. Your OLAP & OLTP workloads should be dominating the compute. ETL ops (loading/set ops) should be a ghost in terms of cpu time and memory. None of this is vendor specific knowledge. Every major engine has a reasonable way to bulk load and perform quick merging of records.
> I would emphasize the importance of batching and set operations.
Please, preach your gospel more loudly and frequently. It always feels like people complain about RDBMSs being slow because they run insert queries one at a time.
> JavaScript is an imperative language that browser wars, framework trends, and open-source maintainer preferences reshaped every few years. It rewards you for keeping up.
> Take a React component from 2015
Javascript is actually fully backwards-compatible, to not break the Web. Any javascript from 10 years ago works in the browser. This is good but also a bit of a burden, since the language can only expand but not shrink.
React is a library, and like all libraries it has breaking versions.
Not understanding the basic difference between the two kinda undermines the credibility of the article.
Also, in a similar way, core, ANSI SQL is largely backwards compatible, but all the SQL dialects linked to various DBMS implementation are generally incompatible. Obviously that's not mentioned in the article.
> Not a tutorial. Not an ORM. Actual SQL: joins, subqueries, window functions, query plans.
Not text written by a human. Not a style that an real writer would ever use. Actual AI slop: Short sentences. Incorrect facts. Not X, Y.
>The Only Programming Language Built on Mathematics, Not Fashion
As a modern array language D4M is the natural successor for SQL [1].
D4M is based on mathematics like SQL, specifically associative array algebra but not relational unlike SQL. It's more generic since can it caters to most modern data abstractions including spreadsheets, database tables, matrices, and graphs [2].
You can achieve 100M database inserts per second with D4M and Accumulo more than a decade ago back in 2014 [3].
[1] D4M: Dynamic Distributed Dimensional Data Model:
https://d4m.mit.edu/
[2] Mathematics of Big Data: Spreadsheets, Databases, Matrices, and Graphs:
https://direct.mit.edu/books/monograph/5691/Mathematics-of-B...
[3] Achieving 100M database inserts per second using Apache Accumulo and D4M (2017 - 46 comments):
https://news.ycombinator.com/item?id=13465141
Sounds interesting, but how can I use it to talk with an Oracle/MySQL/PostgreSQL database?
I feel you missed the point of the article :)
That's true. SQL knowledge is one of the few skills that didn't age.
1. C language.
2. *nix tools (shell and friends).
3. SQL.
4. Basic IPv4 networking.
These things I learned around 20 years ago, they didn't change much and they are useful for me to this day.
I've been slowly transitioning from using an ORM to just plain SQL. It's so much simpler. Less magic, more explicitness, and more control. Also, much better performance. I think the thing is to construct your model around the different queries you need to perform. In many cases, especially a CRUD-type situation, you'll end up with 10-20 different SQL queries, and that's it.
Once you break free of ORM’s I find the code so much simpler to maintain.
Here’s the query(typically multiple different subqueries and return types), here’s the params, give me all the data back and something like Dapper in .net is an absolute godsend to convert it.
Additionally learn stored procedures.
Helps simplify complex SQL queries and no need to waste network traffic on data that client side is never going to use, and waste CPU cycles processing it.
Yes, what about database portability?
I am on my 50s and it only mattered on a single project, which was anyway a middleware for application servers.
> Additionally learn stored procedures.
For sure, but have a solid grounding in set theory to go with it.
I've dealt with so many poorly-performing stored procedures that ended up being written as iteration over a CURSOR when they could have been done with sets. Programmers who don't grok set theory reach for iterative constructs which, while they work fine, are an impedance mismatch with SQL.
- I recently read that most programmers SQL knowledge is outdated by 20 years and it’s true for me. There are quite a lot of features in most DBs that feel very "new" to me.
- Comparing SQL to React weakens the argument. SQL is the language, React is a piece of software. You certainly can run 30 year old JS today in modern browsers.
I’ve been using Postgres for over 6 years (since I started), and I honestly think it’s one of the best investments you can make as a developer
For me SQL has long been the gateway to the world of development. I work in the UK non-profit sector and traditionally this kind of technical knowledge is rare, so for any team I've worked with I've built learning pathways that start with SQL before pushing out into Python, Linux, and other things. We're not exactly at the bleeding edge of current technologies, but SQL has consistently proved to be a great jumping-off point for novices who have even a passing interest in computing.
> Edgar Codd formalised relational algebra in 1970. SQL sits on top of it as a declarative interface. You describe what you want. The database engine decides how to get it. The engine improves every year. Your query stays the same.
Although SQL is of course not relational Algebra (and others like Datalog and D4M are better), it's still cool. It inspired kSQL like Lil uses https://beyondloom.com/decker/lil.html#lilthequerylanguage , which inspired the code I'm most proud of: https://codeberg.org/veqq/declarative-dsls A common query language, a common idiom, for many data structures (arrays, hashmaps, datafremas) is liberating, permitting you to e.g. solve sudoku, make mandelbrot sets or calculate primes directly:
Or e.g. The tests file has many such things (like the sudoku solver) and even datalog and minikanren implemented on top of this!Datalog is the dream. But SQL with a good query builder like Clojure's honeysql is not so bad.
That and SQLite seems to be able to scale to almost any problem, is disgustingly fast and with litestream incredibly resilient.
I've learned SQL around 20 years ago, and in all this time I've felt it was just a poorly designed language. It was always infuriating to write because of its verbose nature. Keywords were split into two words. I'm still shocked it's not "GROUPBY". There is no composition and modularization of logic, queries become massive expressions.
I know I'm in the minority in places like this, but I've spent all my life using ORMs, and never once regretted it. And I'm the kind of person that actually likes low-level C from time to time. SQL just feels like a poor abstraction layer: either go higher or lower.
It’s a good abstraction layer, and a fundamentally good/effecient model of organization and data management. It’s a horrible language, has a meaningless standards doc, some of the worst debugging tooling of modern system and generally any tooling outside of the RDBMS engine itself is 20 years stale.
The only difficult part in arguing this is that RDBMS != SQL != RelationalAlgebra, and it’s very often forgotten
Same can be said for learning an OO programming language or a procedural programming language. I learned C++ at school and started using Java on my first job. I forgot how to work correctly with pointers but I have tried multiple languages (using the same paradigms) and managed to build working software
Everyone knows SQL already. The harder parts that pay off are schema design, knowing how to interact with your DB in code, and knowing all the ins and outs of whatever DBMS you're using.
I would emphasize the importance of batching and set operations. This is where I think many developers lose track of the rabbit, because you don't have much control over either of these things via ORMs. You have to get your hands dirty with raw command text.
The value of this stuff is difficult to overstate. Batching allows for you to rapidly load the RDBMS. The first few times you test, it will probably go so fast you won't believe it loaded anything at all. Set operations allow for you to bring this newly loaded data to visibility in production tables nearly instantly. Your OLAP & OLTP workloads should be dominating the compute. ETL ops (loading/set ops) should be a ghost in terms of cpu time and memory. None of this is vendor specific knowledge. Every major engine has a reasonable way to bulk load and perform quick merging of records.
> I would emphasize the importance of batching and set operations.
Please, preach your gospel more loudly and frequently. It always feels like people complain about RDBMSs being slow because they run insert queries one at a time.
Well yeah they should've banned ORMs in the Geneva Convention. Quickest way to irreversibly ruin your schema design and backend code.
Just, for god's sake, move SELECT after GROUP BY, I beg you.
I've played once with codesignal to pass SQL chapters and it really helped to advance querying skills.
I refuse to learn SQL. I'm not a computer, I'll let them deal with that.
SELECT excuse FROM ignorance ORDER BY snobbery DESC LIMIT 1;
> JavaScript is an imperative language that browser wars, framework trends, and open-source maintainer preferences reshaped every few years. It rewards you for keeping up. > Take a React component from 2015
Javascript is actually fully backwards-compatible, to not break the Web. Any javascript from 10 years ago works in the browser. This is good but also a bit of a burden, since the language can only expand but not shrink. React is a library, and like all libraries it has breaking versions. Not understanding the basic difference between the two kinda undermines the credibility of the article.
Also, in a similar way, core, ANSI SQL is largely backwards compatible, but all the SQL dialects linked to various DBMS implementation are generally incompatible. Obviously that's not mentioned in the article.
> Not a tutorial. Not an ORM. Actual SQL: joins, subqueries, window functions, query plans.
Not text written by a human. Not a style that an real writer would ever use. Actual AI slop: Short sentences. Incorrect facts. Not X, Y.
> Not a tutorial. Not an ORM. Actual SQL: joins, subqueries, window functions, query plans.
My brain absolutely checks out when I read this stuff now.
Not to mention that query plans are absolutely not "actual SQL".
An article laser-targeted at HN's front page, making tantalizingly negative and easily disprovable claims about Javascript? Perish the thought.