The fact that this took 6 years to land stood out to me.
I'm not entirely surprised, and I'm glad the Go team is carefully considering their interfaces before launching. But 6 years seems like a lot of work to land a JSON API.
I'm working on better SIMD support in Dart and one of the motivations for that work is specializing parser specifications based on schema specifications of that language.
Or more concretely, if we had a parser generator + json parser specification that can generate efficient simdjson style parsers, then we should be able to parse json restricted to a subset of json based on a schema at least as efficiently as simdjson can parse general purpose json.
That way we could have the best of both worlds, a (potentially lazy) statically type-safe model and simdjson performance!
I still think that this is definitely doable, it's just that a lot of infrastructure needs to be built for it.
I remember reading that Meta is doing something similar with binary formats. I find it weird that there's nothing like that for json yet.
At this stage to me Go is the new C compiler. Quite honestly I spent 10 years or more writing Go code. At times 18 hours a day. And now with AI, I get models to write my Go code. I STILL want it to be Go code. Is that strange? No because I understand the performance and safety of Go. But I rarely write it directly anymore. Probably not at all in 6-9 months. Yet I still occasionally look at the code to see what the output looks like.
So yea hopefully a nice new addition but only my prompts now matter. So need to make sure they pickup on the new API.
I'm just getting the neuron-band back together where Golang is concerned. I have done some in the past for little system tasks, but I am switching away from interpreted languages entirely for web stuff, and initially to Go, as I switch away from hand-coding to at least LLM-supported coding.
So I am a bit out of date on the pain points (I never really felt pain points when I last used it, because the jobs I was doing had very small, quite Go-friendly shapes).
I generally like `encoding/json/v2` and I'm glad they did it.
I wish they'd left `MarshalIndent` in there though. The equivalent code is super complicated and non-obvious, so it's a pain to migrate, and it's a pain when you're banging out some quick code and want to dump some readable JSON. I suppose these days the LLMs can write the complicated version for us, of course.
the marshal api has been very annoying. why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable. it's certainly a code bug. to me it's like int+str returning an error instead of compiler error
> why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable
Not necessarily. Among other things, MarshalJSON methods called during marshaling can return an error for whatever reason the developer sees fit - one possibility that comes to mind is that part of the object tree being marshalled is lazily loaded, and failed to do so.
And, as others have mentioned, there are types like "any" whose marshalability can't be determined at compile time. Those types aren't necessarily at the top level, either; they could appear anywhere in the object tree.
This isn’t a type system problem. It’s a data origin problem. Ie it could be any random JSON blob a user entered. The compiler doesn’t know this ahead of time.
If you wanted type safety with JSON in Go then don’t use the marshal interface. Period.
For unmarshalling, but the original complaint was about marshalling. And for that the type system could help if the marshal API only allowed you to marshal objects of types that supported marshalling.
i meant i hoped it's panic as opposed to return an error. if there was an error, how am i supposed to handle it? can i tell the user hey sorry my bad it's likely a code error? crash is better than possible eating the error siliently
It is trivial to ignore an error in Go if you know your use case cannot result in one. But at the end of the day, json.Marshal takes an `any` as a parameter, so it must return an error.
The fact that this took 6 years to land stood out to me.
I'm not entirely surprised, and I'm glad the Go team is carefully considering their interfaces before launching. But 6 years seems like a lot of work to land a JSON API.
I'm working on better SIMD support in Dart and one of the motivations for that work is specializing parser specifications based on schema specifications of that language.
Or more concretely, if we had a parser generator + json parser specification that can generate efficient simdjson style parsers, then we should be able to parse json restricted to a subset of json based on a schema at least as efficiently as simdjson can parse general purpose json.
That way we could have the best of both worlds, a (potentially lazy) statically type-safe model and simdjson performance!
I still think that this is definitely doable, it's just that a lot of infrastructure needs to be built for it.
I remember reading that Meta is doing something similar with binary formats. I find it weird that there's nothing like that for json yet.
This approach is called "staged parsing":
https://dl.acm.org/doi/pdf/10.1145/2714064.2660241
https://dl.acm.org/doi/pdf/10.1145/3314221.3314625
At this stage to me Go is the new C compiler. Quite honestly I spent 10 years or more writing Go code. At times 18 hours a day. And now with AI, I get models to write my Go code. I STILL want it to be Go code. Is that strange? No because I understand the performance and safety of Go. But I rarely write it directly anymore. Probably not at all in 6-9 months. Yet I still occasionally look at the code to see what the output looks like.
So yea hopefully a nice new addition but only my prompts now matter. So need to make sure they pickup on the new API.
This is mostly why I have come back to Go.
I am using local LLMs and Go has a bunch of strengths where smaller contexts are concerned; less complicated flow of control being one of them.
I… think this is good?
I'm just getting the neuron-band back together where Golang is concerned. I have done some in the past for little system tasks, but I am switching away from interpreted languages entirely for web stuff, and initially to Go, as I switch away from hand-coding to at least LLM-supported coding.
So I am a bit out of date on the pain points (I never really felt pain points when I last used it, because the jobs I was doing had very small, quite Go-friendly shapes).
I generally like `encoding/json/v2` and I'm glad they did it.
I wish they'd left `MarshalIndent` in there though. The equivalent code is super complicated and non-obvious, so it's a pain to migrate, and it's a pain when you're banging out some quick code and want to dump some readable JSON. I suppose these days the LLMs can write the complicated version for us, of course.
I'm not sure if I'm missing something, but isn't this functionality just moved into jsontext? https://pkg.go.dev/encoding/json/jsontext#Value.Indent
the marshal api has been very annoying. why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable. it's certainly a code bug. to me it's like int+str returning an error instead of compiler error
> why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable
Not necessarily. Among other things, MarshalJSON methods called during marshaling can return an error for whatever reason the developer sees fit - one possibility that comes to mind is that part of the object tree being marshalled is lazily loaded, and failed to do so.
And, as others have mentioned, there are types like "any" whose marshalability can't be determined at compile time. Those types aren't necessarily at the top level, either; they could appear anywhere in the object tree.
Types may be mostly marshallabble, have a subset of values that are unmarshallable. E.g. non-finite floats.
Then write yourself a MustMarshal wrapper that panics.
A generic Must[T] you mean ;)
I wouldn’t class unmarshallable data as a compile time error.
Plus the point of the marshal interface is that it’s reusable.
It could be if go had a more powerful type system.
This isn’t a type system problem. It’s a data origin problem. Ie it could be any random JSON blob a user entered. The compiler doesn’t know this ahead of time.
If you wanted type safety with JSON in Go then don’t use the marshal interface. Period.
For unmarshalling, but the original complaint was about marshalling. And for that the type system could help if the marshal API only allowed you to marshal objects of types that supported marshalling.
i meant i hoped it's panic as opposed to return an error. if there was an error, how am i supposed to handle it? can i tell the user hey sorry my bad it's likely a code error? crash is better than possible eating the error siliently
JSON is a runtime input of unknown type but in your example those are known types so the compiler can catch them.
It is trivial to ignore an error in Go if you know your use case cannot result in one. But at the end of the day, json.Marshal takes an `any` as a parameter, so it must return an error.
do not know at compile time that input is not marshalable, every language marshal must return error for this reason, not only golang