"But accepting the full S3Client here ties UploadReport to an interface that’s too broad. A fake must implement all the methods just to satisfy it."
This isn't really true. Your mock inplementation can embed the interface, but only implement the one required method. Calling the unimplemented methods will panic, but that's not unreasonable for mocks.
At work we use it heavily. You don't really see "a zillion interfaces" after a while, only set of dependencies of a package which is easy to read, and easy to understand.
"makes it hard to cone up with good names" is not really a problem, if you have a `CreateRequest` method you name the interface `RequestCreator`. If you have a request CRUD interface, it's probably a `RequestRepository`.
The benefits outweigh the drawbacks 10 to one. The most rewarding thing about this pattern is how easy it is to split up large implementations, and _keep_ them small.
Any method you forget to overwrite from the embed struct gives a false "impression" you can call any method from mockS3.
Most of time code inside test will be:
// embedded S3Client not properly initialized
mock := mockS3{}
// somewhere inside the business logic
s3.UploadReport(...) // surprise
Go is flexible, you can define a complete interface at producer and consumers still can use their own interface only with required methods if they want.
I revile this pattern. Look at the examples and imagine these are real and everything in the system is abstracted like this, and your coworkers ran out of concise names for their interfaces. Now you have to hop to 7 other files, through abstractions (and then read the DI code to understand which code actually implements this and what it specifically does) and keep all that context in your head… all in service of the I in some stupid acronym, just to build a mental model of what a piece of code does.
Go used to specifically warn against the overuse of this pattern in its teaching documentation, but let me offer an alternative so I’m not just complaining:
Just write functions where the logic is clear to the reader. You’ll thank yourself in 6 months when you’re chasing down a bug
This is a common gripe among former Java programmers who still believe that the point of interfaces is the type hierarchy (and as a result misunderstand Interface Segregation). They hang on to interfaces like they're these precious things that must be given precious names.
Interfaces are not precious. Why would anyone care what their name is? Their actual purpose is to wrap a set of behaviors under a single umbrella. Who cares what the color of the umbrella is? It's locally defined (near the function where the behaviors are used). Before passing an object, just make sure that it has the required methods and you're done. You don't have to be creative about what you name an interface. It does a thing? Call it "ThingDoer".
Also, why would you care to know which code implements a particular interface? It's equivalent to asking give me a list of all types that have this exact set of behavior? I'm possibly being myopic, but I've never considered this of particular importance, at least not as important as being conservative about the behavior you require from dependencies. Having types enumerate all the interfaces they implement is the old school approach (e.g. Java). Go's approach is closer to true Interface Segration. It's done downstream. Just patch the dependency with missing methods. No need to patch the type signature up with needless "implements this, that, other" declarations, which can only create the side-effect that to patch a type from some distant library, you'd have to inherit just so that you can locally declare that you also implement an additional interface. I don't know about you, but to the idea of never having to deal with inheritance in my code ever again I say "good riddance".
Again, interface segregation is about the behavior, not the name. The exact same combination of methods could be defined under a hundred different umbrellas, it would still not matter. If a dependency has the methods, it's good to go.
I have written production Java code, but no production Go code. I think you skipped over the commenter's main point while replying to them: you need to be able to have a good mental model of the code.
A few well defined interfaces have the advantage of being easy to understand and see usages around the codebase without the overhead of many different variants of an interface. This is extremely important if you are not familiar with a given codebase.
I'm not against segregated interfaces, but I feel like over abstracting can result in code that's harder to understand. There's a balance to be had and thought should go into introducing new interfaces, especially when working on a project with many other devs contributing.
I'm a Java dev, so I'm biased. I love being to easily understand and reason about the type system. I understand that an interface is about a set of behaviors, but when I've worked with Go code I've found it much more difficult to get my IDE to point out all the different ways some interface could be implemented. I see the advantages that Go style interfaces bring, but I personally find it harder to keep a mental model when working with Go.
I generally advise to avoid introducing interfaces strictly for testing. Instead, design the data types themselves to be testable and only use interfaces when you expect to need differing implementations. ie, avoid premature abstraction and you get rid of a whole class of problems.
For example, if you only use S3, it is premature abstraction to accept an interface for something that may not be S3. Just accept the S3 client itself as input.
Then the S3 client can be designed to be testable by itself by having the lowest-level dependencies (ie, network calls) stubbed out. For example, it can take a fake implementation that has hard-coded S3 URLs mapped to blobs. Everything that tests code with S3 simply has to pre-populate a list of URLs and blobs they need for the test, which itself can be centralized or distributed as necessary depending on the way the code is organized.
Generally, I/O is great level to use an interface and to stub out. Network, disk, etc. Then if you have good dependency injection practicies, it becomes fairly easy to use real structs in testing and to avoid interfaces purely for testing.
I agree with not introducing abstractions prematurely, but your suggestion hinges on the design of the S3 client. In practice, if your code is depending on a library you have no control over, you'll have to work with interfaces if you want to prevent your tests from doing I/O. So in unit tests you can pass an in-memory mock/stub, and in integration tests, you can pass a real S3 client, and connect to a real S3 server running locally.
So I don't see dependency injection with interfaces as being premature abstractions. You're simply explicitly specifying the API your code depends on, instead of depending on a concrete type of which you might only use one or two methods. I think this is a good pattern to follow in general, with no practical drawbacks.
Yes, this is absolutely dependent on the design S3 client.
The reality of development is we have to merge different design philosophies into one code base. Things can get messy. 100% agreed.
The approach I advocate for is more for a) organizing the code you do own, and b) designing in a way that you play nice with others who may import your code.
Rather than defining all these one-method interfaces, why not specify a function type?
Instead of
type Saver interface {
Save(data []byte) error
}
You could have
type saves func([]byte) error
Seems less bulky than an interface, more concise to mock too.
It's more effort when you need to "promote" the port / input type to a full interface, but I think that's a reasonable tradeoff to avoid callers of your function constantly creating structs just to hang methods off
I don't care that much about defining a minimal interface or whether the producer or consumer defines it. the pain point for me is when you start passing interfaces up and down the stack and they become impossible to trace back to the concrete type. If you take an interface you should use it directly and avoid passing it down to another one of your dependencies. This keeps the layers you need to jump through to find the concrete type to a minimum.
> However, there’s still one issue: Backup only calls Save, yet the Storage interface includes both Save and Load. If Storage later gains more methods, every fake must grow too, even if those methods aren’t used.
First, why would you ever add methods to a public interface? Second, the next version of the Backup's implementation might very well want to call Load as well (e.g. for deduplication purposes) and then you suddenly need to add more methods to your fakes anyhow.
In the end, it really depends on who owns FileStorage and Backup: if it's the same team/person, the ISP is immaterial. If they are different, then yes, the owner of Backup() would be better served by declaring a Storage interface of their own and delegate the job of writing adapters that make e.g. FileStorage to conform to it to the users of Backup() method.
If you add a method to an interface, you break every source file that uses a concrete type in place of the interface (ie, passes a struct to a function that takes an interface) unless you also update all the concrete types to implement the new method (or you update them embed the interface, which is yucky).
For a public interface, you have to track down all the clients, which may be infeasible, especially in an open ecosystem.
At $WORK we have taken interface segregation to the extreme. For example, say we have a data access object that gets consumed by many different packages. Rather than defining a single interface and mock on the producer side that can be reused by all these packages, each package defines its own minimal interface containing only the methods it needs, and a corresponding mock. This makes it extremely difficult to trace the execution flow, and turns a simple function signature change into an hour-long ordeal of regenerating mocks.
> a single interface and mock on the producer side
I still believe in Go it is better to _start_ with interfaces on the consumer and focus on "what you need" with interfaces instead of "what you provide" since there's no "implements" concept.
I get the mock argument all the time for having producer interfaces and I don't deny at a certain scale it makes sense but I don't understand why so many people reach for it out of the gate.
I'm genuinely curious if you have felt the pain from interfaces on the producer that would go away if there were just (multiple?) concrete types in use or if you happen to have a notion of OO in Go that is hard to let go of?
> or if you happen to have a notion of OO in Go that is hard to let go of?
So much this. I think Go's interfaces are widely misunderstood. Often times when they're complained about, it boils down to "<old OO language> did interface this way. Why Go won't abide?" There's insistence in turning them into cherished pets. Vastly more treasured than they ought to be in Go, a meaningless thin paper wrapper that says "I require these behaviors".
> Rather than defining a single interface and mock on the producer side that can be reused by all these packages
This is the answer. The domain that exports the API should also provide a high fidelity test double that is a fake/in memory implementation (not a mock!) that all internal downstream consumers can use.
New method on the interface (or behavioral change to existing methods)? Update the fake in the same change (you have to, otherwise the fake won't meet the interface and uses won't compile!), and your build system can run all tests that use it.
> The domain that exports the API should also provide a high fidelity test double that is a fake/in memory implementation (not a mock!)
Not a mock? But that's exactly what a mock is: An implementation that isn't authentic, but that doesn't try to deceive. In other words, something that behaves just like the "real thing" (to the extent that matters), but is not authentically the "real thing". Hence the name.
There are different definitions of the term "mock". You described the generic usage where "mock" is a catch-all for "not the real thing", but there are several terms in this space to refer to more precise concepts.
What I've seen:
* "test double" - a catch-all term for "not the real thing". What you called a "mock". But this phrasing is more general so the term "mock" can be used elsewhere.
* "fake" - a simplified implementation, complex enough to mimic real behavior. It probably uses a lot of the real thing under the hood, but with unnecessary testing-related features removed. ie: a real database that only runs in memory.
* "stub" - a very thin shim that only provides look-up style responses. Basically a map of which inputs produce which outputs.
* "mock" - an object that has expectations about how it is to be used. It encodes some test logic itself.
The Go ecosystem seems to prefer avoiding test objects that encode expectations about how they are used and the community uses the term "mock" specifically to refer to that. This is why you hear "don't use mocks in Go". It refers to a specific type of test double.
By these definitions, OP was referring to a "fake". And I agree with OP that there is much benefit to providing canonical test fakes, so long as you don't lock users into only using your test fake because it will fall short of someone's needs at some point.
Unfortunately there's no authoritative source for these terms (that I'm aware of), so there's always arguing about what exactly words mean.
My best guess is that software development co-opted the term "mock" from the vocabulary of other fields, and the folks who were into formalities used the term for a more specific definition, but the software dev discipline doesn't follow much formal vocabulary and a healthy portion of devs intuitively use the term "mock" generically. (I myself was in the field for years before I encountered any formal vocabulary on the topic.)
I 100% agree with what you've written, but if you haven't checked it out, I'll highly suggest trying mockery v3 for mocks: https://vektra.github.io/mockery
It's generally faster than a build (no linking steps), regardless of the number of things to generate, because it loads types just once and generates everything needed from that. Wildly better than the go:generate based ones.
AFAICT that uses go/types, loaded uniquely per execution via packages.Load¹, which is by far the primary reason why e.g. go.uber.org/mock (previously github.com/golang/mock) can become extremely slow.
mockery v3 does not do this. it type-checks just once for ALL mocks, regardless of the number, so it essentially does not grow slower as you create more mocks (since type checking is usually FAR slower than producing the mock).
Yes, this is exactly the problem with go's recipe.
Either you copypaste the same interface over and over and over, with the maintenance nightmare that is, or you always have these struct-and-interface pairs, where it's unclear why there is an interface to begin with. If the answer is testing, maybe that's the wrong question ti begin with.
So, I would rather have duck typing (the structural kind, not just interfaces) for easy testing. I wonder if it would technically be possible to only compile with duck typing in test, in a hypothetical language.
> I wonder if it would technically be possible to only compile with duck typing in test
Not exactly the same thing, but you can use build tags to compile with a different implementation for a concrete type while under test.
Sounds like a serious case of overthinking it, though. The places where you will justifiably swap implementations during testing are also places where you will justifiably want to be able to swap implementations in general. That's what interfaces are there for.
If you cannot find any reason why you'd benefit from a second implementation outside of the testing scenario, you won't need it while under test either. In that case, learn how to test properly and use the single implementation you already have under all scenarios.
> The places where you will justifiably swap implementations during testing are also places where you will justifiably want to be able to swap implementations in general.
I don't get this. Just because I want to mock something doesn't mean I really need different implementations. That was my point: if I could just duck-type-swap it in a test, it would be so much easier than 1. create an interface that just repeats all methods, and then 2. need to use some mock generation tool.
If I don't mock it, then my tests become integration test behemoths. Which have their use too, but it's bad if you can't write simple unit tests anymore.
> then my tests become integration test behemoths.
There are no consistent definitions found in the world of testing, but I assume integration here means entry into some kind of third-party system that you don't have immediate control over? That seems to be how it is most commonly used. And that's exactly one of the places you'd benefit from enabling multiple implementations, even if testing wasn't in the picture. There are many reasons why you don't want to couple your application to these integrations. The benefits found under test are a manifestation of the very same, not some unique situation.
What is the alternative though? In strongly typed languages like Go, Rust, etc.. you must define the contract. So you either focus on what you need, or you just make a kitchen-sink interface.
I don't even want to think about the global or runtime rewriting that is possible (common) in Java and JavaScript as a reasonable solution to this DI problem.
I'm still fiddling with this so I haven't seen it at scale yet, but in some code I'm writing now, I have a centralized repository for services that register themselves. There is a struct that will provide the union of all possible subservices that they may require (logging, caching, db, etc.). The service registers a function with the central repository that can take that object, but can also take an interface that it defines with just a subset of the values.
This uses reflect and is nominally checked at run time, but over time more and more I am distinguishing between a runtime check that runs arbitrarily often over the execution of a program, and one that runs in an init phase. I have a command-line option on the main executable that runs the initialization without actually starting any services up, so even though it's a run-time panic if a service misregisters itself, it's caught at commit time in my pre-commit hook. (I am also moving towards worrying less about what is necessarily caught at "compile time" and what is caught at commit time, which opens up some possibilities in any language.)
The central service module also defines some convenient one-method interfaces that the services can use, so one service may look like:
type myDependencies interface {
services.UsesDB
services.UsesLogging
}
func init() {
services.Register(func(in myDependencies) error {
// init here
}
}
and another may have
type myDependencies interface {
services.UsesLogging
services.UsesCaching
services.UsesWebCrawler
}
// func init() { etc. }
and in this way, each services declaring its own dependencies means each service's test cases only need to worry about what it actually uses, and the interfaces don't pollute anything else. This fully decouples "the set of services I'm providing from my modules" from "the services each module requires", and while I don't get compile-time checking that a module's service requirements are satisfied, I can easily get commit-time checking.
I also have some default fakes that things can use, but they're not necessary. They're just one convenient implementation for testing if you need them.
tbh this sounds pretty similar to go.uber.org/fx (or dig). or really almost any dependency injection framework, though e.g. wire is compile-time validated rather than run-time (and thus much harder for some kinds of runtime flexibility - I make no claim to one being better than the other).
DI frameworks, when they're not gigantic monstrosities like in Java, are pretty great.
Yes. The nice thing about this is that it's one function, about 20-30 lines, rather than a "framework".
I've been operating up to this point without this structure in a fairly similar manner, and it has worked fine in the tens-of-thousands-of-lines range. I can see maybe another order or two up I'd need more structure, but people really badly underestimate the costs of these massive frameworks, IMHO, and also often fail to understand that the value proposition of these frameworks often just boils down to something that could fit comfortably in the aforementioned 20-30 lines.
yeah, if it's only 20-30 lines then it's almost certainly overkill to do any way except by hand.
most of the stuff I've done has involved at least 20-30 libraries, many of which have other dependencies and config, so it's on the order of hundreds or thousands of lines if written by hand. it's totally worth a (simple) DI tool at that point.
"But accepting the full S3Client here ties UploadReport to an interface that’s too broad. A fake must implement all the methods just to satisfy it."
In NET, one would simply mock one or two methods required by the implementation under the test. If I'm using Moq, then one would set it up in strict mode, to avoid surprises if unit under test starts calling something it didn't before.
> Object-oriented (OO) patterns get a lot of flak in the Go community, and often for good reason.
This isn't really an OO pattern, as the rest of the post demonstrates. It's just a pattern that applies across most any language where you can make a distinction between an interface/typeclass or whatever, and a concrete type.
> distinction between an interface/typeclass or whatever, and a concrete type.
This is the essence of OOP.
"The notion of an interface is what truly characterizes objects - not classes, not inheritance, not mutable state. Read William Cook's classic essay for a deep discussion on this." - Gilad Bracha
> The notion of an interface is what truly characterizes objects
Objects, but not OO. OO takes the concept further — what it calls message passing — which allows an object to dynamically respond to messages at runtime, even where the message does not conform to any known interface.
Not quite. With OO, there is no set. An object always responds to all messages, even when the message contains arbitrary garbage. An object can respond with "I don't understand" when faced with garbage, which is a common pattern in OO languages, but it doesn't have to. An object could equally respond with a value of 1 if it wants.
Dynamic typing is a necessary precondition for OO[1], but that is not what defines it. Javascript, for example, has objects and is dynamically typed, but is not OO. If I call object.random_gibberish in Javascript, the object will never know. The runtime will blow up before it ever finds out. Whereas in an OO language the object will receive a message containing "random_gibberish" and it can decide what it do with it.
[1] Objective-C demonstrated that you can include static-typing in a partial, somewhat hacky way, but there is no way to avoid dynamic-typing completely.
It is not. The only languages (that people have actually heard of, at least) that are OO are Smalltalk, Ruby, and Objective-C. Swift also includes OO features, enabled with the @objc directive, for the sake of backwards compatibility with Objective-C, but "Swift proper" has tried to distance itself from the concept.
Go channels share some basic conceptual ideas with message passing, but they don't go far enough to bear any direct resemblance to OO; most notably they are not tied to objects in any way.
How so? Genuine question. Duck typing is “try it and see if it supports an action”, where interface declaration is the opposite: declare what methods must be supported by what you interact with.
Sure, type checking in Python (Protocols or not) is done very differently and less strongly than in Go, but the semantic pattern of interface segregation seems to be equivalently possible in both languages—and very different from duck typing.
Duck typing is often equated with structural typing. You’re right that officially (at least according to Wikipedia) duck typing is dynamic, while structural is the same idea, but static.
Either way, the thing folks are contrasting with here is nominal typing of interfaces, where a type explicitly declares which interfaces it implements. In Go it’s “if it quacks like a duck, it’s a duck”, just statically checked.
I'm saying that at some point declaring the minimal interface a caller uses, for example Reader and Writer instead of a concrete FS type, starts to look like duck typing. In python a functions use of v.read() or v.write() defines what v should provide.
In Go it is compile time and Python it is runtime, but it is similar.
In Python (often) you don't care about the type of v just that it implements v.write() and in an interface based separation of API concerns you declare that v.write() is provided by the interface.
The aim is the same, duck typing or interfaces. And the outcome benefits are the same, at runtime or compile time.
Also yes Protocols can be used to type check quacks, bringing it more inline with the Go examples in the blog.
However my point is more from a SOLID perspective duck typing and minimal dependency interfaces sort of achieve similar ends... Minimal dependency and assumption by calling code.
Except you need a typed variable that implements the interface or you need to cast an any into an interface type. If the "any" type implemented all interfaces then it would be duck typing, but since the language enforces types at the call level, it is not.
"But accepting the full S3Client here ties UploadReport to an interface that’s too broad. A fake must implement all the methods just to satisfy it."
This isn't really true. Your mock inplementation can embed the interface, but only implement the one required method. Calling the unimplemented methods will panic, but that's not unreasonable for mocks.
That is:
You don't have to implement all the other methods.Defining a zillion interfaces, all the permutations of methods in use, makes it hard to cone up with good names, and thus hard to read.
While you can do that, having unused methods that don't work is a footgun. It's cleaner if they don't exist at all.
Is this pattern commonly used? Any drawbacks?
Sounds much better than the interface boilerplate if it's just for the sake of testing.
At work we use it heavily. You don't really see "a zillion interfaces" after a while, only set of dependencies of a package which is easy to read, and easy to understand.
"makes it hard to cone up with good names" is not really a problem, if you have a `CreateRequest` method you name the interface `RequestCreator`. If you have a request CRUD interface, it's probably a `RequestRepository`.
The benefits outweigh the drawbacks 10 to one. The most rewarding thing about this pattern is how easy it is to split up large implementations, and _keep_ them small.
Any method you forget to overwrite from the embed struct gives a false "impression" you can call any method from mockS3. Most of time code inside test will be:
Go is flexible, you can define a complete interface at producer and consumers still can use their own interface only with required methods if they want.I revile this pattern. Look at the examples and imagine these are real and everything in the system is abstracted like this, and your coworkers ran out of concise names for their interfaces. Now you have to hop to 7 other files, through abstractions (and then read the DI code to understand which code actually implements this and what it specifically does) and keep all that context in your head… all in service of the I in some stupid acronym, just to build a mental model of what a piece of code does.
Go used to specifically warn against the overuse of this pattern in its teaching documentation, but let me offer an alternative so I’m not just complaining: Just write functions where the logic is clear to the reader. You’ll thank yourself in 6 months when you’re chasing down a bug
This is a common gripe among former Java programmers who still believe that the point of interfaces is the type hierarchy (and as a result misunderstand Interface Segregation). They hang on to interfaces like they're these precious things that must be given precious names.
Interfaces are not precious. Why would anyone care what their name is? Their actual purpose is to wrap a set of behaviors under a single umbrella. Who cares what the color of the umbrella is? It's locally defined (near the function where the behaviors are used). Before passing an object, just make sure that it has the required methods and you're done. You don't have to be creative about what you name an interface. It does a thing? Call it "ThingDoer".
Also, why would you care to know which code implements a particular interface? It's equivalent to asking give me a list of all types that have this exact set of behavior? I'm possibly being myopic, but I've never considered this of particular importance, at least not as important as being conservative about the behavior you require from dependencies. Having types enumerate all the interfaces they implement is the old school approach (e.g. Java). Go's approach is closer to true Interface Segration. It's done downstream. Just patch the dependency with missing methods. No need to patch the type signature up with needless "implements this, that, other" declarations, which can only create the side-effect that to patch a type from some distant library, you'd have to inherit just so that you can locally declare that you also implement an additional interface. I don't know about you, but to the idea of never having to deal with inheritance in my code ever again I say "good riddance".
Again, interface segregation is about the behavior, not the name. The exact same combination of methods could be defined under a hundred different umbrellas, it would still not matter. If a dependency has the methods, it's good to go.
I have written production Java code, but no production Go code. I think you skipped over the commenter's main point while replying to them: you need to be able to have a good mental model of the code.
A few well defined interfaces have the advantage of being easy to understand and see usages around the codebase without the overhead of many different variants of an interface. This is extremely important if you are not familiar with a given codebase.
I'm not against segregated interfaces, but I feel like over abstracting can result in code that's harder to understand. There's a balance to be had and thought should go into introducing new interfaces, especially when working on a project with many other devs contributing.
I'm a Java dev, so I'm biased. I love being to easily understand and reason about the type system. I understand that an interface is about a set of behaviors, but when I've worked with Go code I've found it much more difficult to get my IDE to point out all the different ways some interface could be implemented. I see the advantages that Go style interfaces bring, but I personally find it harder to keep a mental model when working with Go.
You’re decreasing coupling at the cost of introducing more entities, and a different sort of complexity, into your system.
Sometimes it’s absolutely worth it. Sometimes not.
"The Tyranny of nouns" when everything has a subtly different name in every context
I generally advise to avoid introducing interfaces strictly for testing. Instead, design the data types themselves to be testable and only use interfaces when you expect to need differing implementations. ie, avoid premature abstraction and you get rid of a whole class of problems.
For example, if you only use S3, it is premature abstraction to accept an interface for something that may not be S3. Just accept the S3 client itself as input.
Then the S3 client can be designed to be testable by itself by having the lowest-level dependencies (ie, network calls) stubbed out. For example, it can take a fake implementation that has hard-coded S3 URLs mapped to blobs. Everything that tests code with S3 simply has to pre-populate a list of URLs and blobs they need for the test, which itself can be centralized or distributed as necessary depending on the way the code is organized.
Generally, I/O is great level to use an interface and to stub out. Network, disk, etc. Then if you have good dependency injection practicies, it becomes fairly easy to use real structs in testing and to avoid interfaces purely for testing.
Related reading from the Google style guide, but focused specifically on the transport layer: https://google.github.io/styleguide/go/best-practices.html#u...
I agree with not introducing abstractions prematurely, but your suggestion hinges on the design of the S3 client. In practice, if your code is depending on a library you have no control over, you'll have to work with interfaces if you want to prevent your tests from doing I/O. So in unit tests you can pass an in-memory mock/stub, and in integration tests, you can pass a real S3 client, and connect to a real S3 server running locally.
So I don't see dependency injection with interfaces as being premature abstractions. You're simply explicitly specifying the API your code depends on, instead of depending on a concrete type of which you might only use one or two methods. I think this is a good pattern to follow in general, with no practical drawbacks.
Yes, this is absolutely dependent on the design S3 client.
The reality of development is we have to merge different design philosophies into one code base. Things can get messy. 100% agreed.
The approach I advocate for is more for a) organizing the code you do own, and b) designing in a way that you play nice with others who may import your code.
Rather than defining all these one-method interfaces, why not specify a function type?
Instead of
You could have Seems less bulky than an interface, more concise to mock too.It's more effort when you need to "promote" the port / input type to a full interface, but I think that's a reasonable tradeoff to avoid callers of your function constantly creating structs just to hang methods off
I don't care that much about defining a minimal interface or whether the producer or consumer defines it. the pain point for me is when you start passing interfaces up and down the stack and they become impossible to trace back to the concrete type. If you take an interface you should use it directly and avoid passing it down to another one of your dependencies. This keeps the layers you need to jump through to find the concrete type to a minimum.
> However, there’s still one issue: Backup only calls Save, yet the Storage interface includes both Save and Load. If Storage later gains more methods, every fake must grow too, even if those methods aren’t used.
First, why would you ever add methods to a public interface? Second, the next version of the Backup's implementation might very well want to call Load as well (e.g. for deduplication purposes) and then you suddenly need to add more methods to your fakes anyhow.
In the end, it really depends on who owns FileStorage and Backup: if it's the same team/person, the ISP is immaterial. If they are different, then yes, the owner of Backup() would be better served by declaring a Storage interface of their own and delegate the job of writing adapters that make e.g. FileStorage to conform to it to the users of Backup() method.
>First, why would you ever add methods to a public interface?
In the go world, it's a little more acceptable to do that versus something like Java because you're really not going to break anything
If you add a method to an interface, you break every source file that uses a concrete type in place of the interface (ie, passes a struct to a function that takes an interface) unless you also update all the concrete types to implement the new method (or you update them embed the interface, which is yucky).
For a public interface, you have to track down all the clients, which may be infeasible, especially in an open ecosystem.
Scheme taught me that OOP is the poor man's closure.
At $WORK we have taken interface segregation to the extreme. For example, say we have a data access object that gets consumed by many different packages. Rather than defining a single interface and mock on the producer side that can be reused by all these packages, each package defines its own minimal interface containing only the methods it needs, and a corresponding mock. This makes it extremely difficult to trace the execution flow, and turns a simple function signature change into an hour-long ordeal of regenerating mocks.
> a single interface and mock on the producer side
I still believe in Go it is better to _start_ with interfaces on the consumer and focus on "what you need" with interfaces instead of "what you provide" since there's no "implements" concept.
I get the mock argument all the time for having producer interfaces and I don't deny at a certain scale it makes sense but I don't understand why so many people reach for it out of the gate.
I'm genuinely curious if you have felt the pain from interfaces on the producer that would go away if there were just (multiple?) concrete types in use or if you happen to have a notion of OO in Go that is hard to let go of?
> or if you happen to have a notion of OO in Go that is hard to let go of?
So much this. I think Go's interfaces are widely misunderstood. Often times when they're complained about, it boils down to "<old OO language> did interface this way. Why Go won't abide?" There's insistence in turning them into cherished pets. Vastly more treasured than they ought to be in Go, a meaningless thin paper wrapper that says "I require these behaviors".
> Rather than defining a single interface and mock on the producer side that can be reused by all these packages
This is the answer. The domain that exports the API should also provide a high fidelity test double that is a fake/in memory implementation (not a mock!) that all internal downstream consumers can use.
New method on the interface (or behavioral change to existing methods)? Update the fake in the same change (you have to, otherwise the fake won't meet the interface and uses won't compile!), and your build system can run all tests that use it.
> The domain that exports the API should also provide a high fidelity test double that is a fake/in memory implementation (not a mock!)
Not a mock? But that's exactly what a mock is: An implementation that isn't authentic, but that doesn't try to deceive. In other words, something that behaves just like the "real thing" (to the extent that matters), but is not authentically the "real thing". Hence the name.
There are different definitions of the term "mock". You described the generic usage where "mock" is a catch-all for "not the real thing", but there are several terms in this space to refer to more precise concepts.
What I've seen:
* "test double" - a catch-all term for "not the real thing". What you called a "mock". But this phrasing is more general so the term "mock" can be used elsewhere.
* "fake" - a simplified implementation, complex enough to mimic real behavior. It probably uses a lot of the real thing under the hood, but with unnecessary testing-related features removed. ie: a real database that only runs in memory.
* "stub" - a very thin shim that only provides look-up style responses. Basically a map of which inputs produce which outputs.
* "mock" - an object that has expectations about how it is to be used. It encodes some test logic itself.
The Go ecosystem seems to prefer avoiding test objects that encode expectations about how they are used and the community uses the term "mock" specifically to refer to that. This is why you hear "don't use mocks in Go". It refers to a specific type of test double.
By these definitions, OP was referring to a "fake". And I agree with OP that there is much benefit to providing canonical test fakes, so long as you don't lock users into only using your test fake because it will fall short of someone's needs at some point.
Unfortunately there's no authoritative source for these terms (that I'm aware of), so there's always arguing about what exactly words mean.
Martin Fowler's definitions are closely aligned with the Go community I'm familiar with: https://martinfowler.com/articles/mocksArentStubs.html
Wikipedia has chosen to cite him as well: https://en.wikipedia.org/wiki/Test_double#General .
My best guess is that software development co-opted the term "mock" from the vocabulary of other fields, and the folks who were into formalities used the term for a more specific definition, but the software dev discipline doesn't follow much formal vocabulary and a healthy portion of devs intuitively use the term "mock" generically. (I myself was in the field for years before I encountered any formal vocabulary on the topic.)
I 100% agree with what you've written, but if you haven't checked it out, I'll highly suggest trying mockery v3 for mocks: https://vektra.github.io/mockery
It's generally faster than a build (no linking steps), regardless of the number of things to generate, because it loads types just once and generates everything needed from that. Wildly better than the go:generate based ones.
Try https://github.com/maxbrunsfeld/counterfeiter
AFAICT that uses go/types, loaded uniquely per execution via packages.Load¹, which is by far the primary reason why e.g. go.uber.org/mock (previously github.com/golang/mock) can become extremely slow.
mockery v3 does not do this. it type-checks just once for ALL mocks, regardless of the number, so it essentially does not grow slower as you create more mocks (since type checking is usually FAR slower than producing the mock).
1: https://github.com/maxbrunsfeld/counterfeiter/blob/000b82ca1...
Yes, this is exactly the problem with go's recipe.
Either you copypaste the same interface over and over and over, with the maintenance nightmare that is, or you always have these struct-and-interface pairs, where it's unclear why there is an interface to begin with. If the answer is testing, maybe that's the wrong question ti begin with.
So, I would rather have duck typing (the structural kind, not just interfaces) for easy testing. I wonder if it would technically be possible to only compile with duck typing in test, in a hypothetical language.
> I wonder if it would technically be possible to only compile with duck typing in test
Not exactly the same thing, but you can use build tags to compile with a different implementation for a concrete type while under test.
Sounds like a serious case of overthinking it, though. The places where you will justifiably swap implementations during testing are also places where you will justifiably want to be able to swap implementations in general. That's what interfaces are there for.
If you cannot find any reason why you'd benefit from a second implementation outside of the testing scenario, you won't need it while under test either. In that case, learn how to test properly and use the single implementation you already have under all scenarios.
> The places where you will justifiably swap implementations during testing are also places where you will justifiably want to be able to swap implementations in general.
I don't get this. Just because I want to mock something doesn't mean I really need different implementations. That was my point: if I could just duck-type-swap it in a test, it would be so much easier than 1. create an interface that just repeats all methods, and then 2. need to use some mock generation tool.
If I don't mock it, then my tests become integration test behemoths. Which have their use too, but it's bad if you can't write simple unit tests anymore.
> then my tests become integration test behemoths.
There are no consistent definitions found in the world of testing, but I assume integration here means entry into some kind of third-party system that you don't have immediate control over? That seems to be how it is most commonly used. And that's exactly one of the places you'd benefit from enabling multiple implementations, even if testing wasn't in the picture. There are many reasons why you don't want to couple your application to these integrations. The benefits found under test are a manifestation of the very same, not some unique situation.
What is the alternative though? In strongly typed languages like Go, Rust, etc.. you must define the contract. So you either focus on what you need, or you just make a kitchen-sink interface.
I don't even want to think about the global or runtime rewriting that is possible (common) in Java and JavaScript as a reasonable solution to this DI problem.
I'm still fiddling with this so I haven't seen it at scale yet, but in some code I'm writing now, I have a centralized repository for services that register themselves. There is a struct that will provide the union of all possible subservices that they may require (logging, caching, db, etc.). The service registers a function with the central repository that can take that object, but can also take an interface that it defines with just a subset of the values.
This uses reflect and is nominally checked at run time, but over time more and more I am distinguishing between a runtime check that runs arbitrarily often over the execution of a program, and one that runs in an init phase. I have a command-line option on the main executable that runs the initialization without actually starting any services up, so even though it's a run-time panic if a service misregisters itself, it's caught at commit time in my pre-commit hook. (I am also moving towards worrying less about what is necessarily caught at "compile time" and what is caught at commit time, which opens up some possibilities in any language.)
The central service module also defines some convenient one-method interfaces that the services can use, so one service may look like:
and another may have and in this way, each services declaring its own dependencies means each service's test cases only need to worry about what it actually uses, and the interfaces don't pollute anything else. This fully decouples "the set of services I'm providing from my modules" from "the services each module requires", and while I don't get compile-time checking that a module's service requirements are satisfied, I can easily get commit-time checking.I also have some default fakes that things can use, but they're not necessary. They're just one convenient implementation for testing if you need them.
tbh this sounds pretty similar to go.uber.org/fx (or dig). or really almost any dependency injection framework, though e.g. wire is compile-time validated rather than run-time (and thus much harder for some kinds of runtime flexibility - I make no claim to one being better than the other).
DI frameworks, when they're not gigantic monstrosities like in Java, are pretty great.
Yes. The nice thing about this is that it's one function, about 20-30 lines, rather than a "framework".
I've been operating up to this point without this structure in a fairly similar manner, and it has worked fine in the tens-of-thousands-of-lines range. I can see maybe another order or two up I'd need more structure, but people really badly underestimate the costs of these massive frameworks, IMHO, and also often fail to understand that the value proposition of these frameworks often just boils down to something that could fit comfortably in the aforementioned 20-30 lines.
yeah, if it's only 20-30 lines then it's almost certainly overkill to do any way except by hand.
most of the stuff I've done has involved at least 20-30 libraries, many of which have other dependencies and config, so it's on the order of hundreds or thousands of lines if written by hand. it's totally worth a (simple) DI tool at that point.
Maybe your actual issue is needing to mock stuff for tests to begin with. Break them down further so they can actually be tested in isolation instead.
"But accepting the full S3Client here ties UploadReport to an interface that’s too broad. A fake must implement all the methods just to satisfy it."
In NET, one would simply mock one or two methods required by the implementation under the test. If I'm using Moq, then one would set it up in strict mode, to avoid surprises if unit under test starts calling something it didn't before.
> Object-oriented (OO) patterns get a lot of flak in the Go community, and often for good reason.
This isn't really an OO pattern, as the rest of the post demonstrates. It's just a pattern that applies across most any language where you can make a distinction between an interface/typeclass or whatever, and a concrete type.
> distinction between an interface/typeclass or whatever, and a concrete type.
This is the essence of OOP.
"The notion of an interface is what truly characterizes objects - not classes, not inheritance, not mutable state. Read William Cook's classic essay for a deep discussion on this." - Gilad Bracha
https://blog.bracha.org/primordialsoup.html?snapshot=Amplefo...
http://www.cs.utexas.edu/~wcook/Drafts/2009/essay.pdf
> The notion of an interface is what truly characterizes objects
Objects, but not OO. OO takes the concept further — what it calls message passing — which allows an object to dynamically respond to messages at runtime, even where the message does not conform to any known interface.
The object has a known interface in this case. It's just not statically defined. Its interface is the set of messages that it responds to.
Not quite. With OO, there is no set. An object always responds to all messages, even when the message contains arbitrary garbage. An object can respond with "I don't understand" when faced with garbage, which is a common pattern in OO languages, but it doesn't have to. An object could equally respond with a value of 1 if it wants.
Dynamic typing is a necessary precondition for OO[1], but that is not what defines it. Javascript, for example, has objects and is dynamically typed, but is not OO. If I call object.random_gibberish in Javascript, the object will never know. The runtime will blow up before it ever finds out. Whereas in an OO language the object will receive a message containing "random_gibberish" and it can decide what it do with it.
[1] Objective-C demonstrated that you can include static-typing in a partial, somewhat hacky way, but there is no way to avoid dynamic-typing completely.
golang allows for the same
The ironic thing is that golang itself is OO.
It is not. The only languages (that people have actually heard of, at least) that are OO are Smalltalk, Ruby, and Objective-C. Swift also includes OO features, enabled with the @objc directive, for the sake of backwards compatibility with Objective-C, but "Swift proper" has tried to distance itself from the concept.
Go channels share some basic conceptual ideas with message passing, but they don't go far enough to bear any direct resemblance to OO; most notably they are not tied to objects in any way.
See also https://news.ycombinator.com/item?id=36908369 (“The bigger the interface, the weaker the abstraction")
Follow the trail of the blog post and you end up with Python and duck typing, and all the foot guns there too.
How so? Genuine question. Duck typing is “try it and see if it supports an action”, where interface declaration is the opposite: declare what methods must be supported by what you interact with.
In Python, that would be a Protocol (https://typing.python.org/en/latest/spec/protocol.html), which is a newer and leas commonly used feature than full, un-annotated duck typing.
Sure, type checking in Python (Protocols or not) is done very differently and less strongly than in Go, but the semantic pattern of interface segregation seems to be equivalently possible in both languages—and very different from duck typing.
Duck typing is often equated with structural typing. You’re right that officially (at least according to Wikipedia) duck typing is dynamic, while structural is the same idea, but static.
Either way, the thing folks are contrasting with here is nominal typing of interfaces, where a type explicitly declares which interfaces it implements. In Go it’s “if it quacks like a duck, it’s a duck”, just statically checked.
I'm saying that at some point declaring the minimal interface a caller uses, for example Reader and Writer instead of a concrete FS type, starts to look like duck typing. In python a functions use of v.read() or v.write() defines what v should provide.
In Go it is compile time and Python it is runtime, but it is similar.
In Python (often) you don't care about the type of v just that it implements v.write() and in an interface based separation of API concerns you declare that v.write() is provided by the interface.
The aim is the same, duck typing or interfaces. And the outcome benefits are the same, at runtime or compile time.
Also yes Protocols can be used to type check quacks, bringing it more inline with the Go examples in the blog.
However my point is more from a SOLID perspective duck typing and minimal dependency interfaces sort of achieve similar ends... Minimal dependency and assumption by calling code.
> starts to look like duck typing.
Except you need a typed variable that implements the interface or you need to cast an any into an interface type. If the "any" type implemented all interfaces then it would be duck typing, but since the language enforces types at the call level, it is not.