The Experiment

I recently tried an experiment using OpenAI's ChatGPT to design a simple OpenAPI specification. My aim was to exercise the following workflow which reflects my typical manual design process:

• Start with a JSON Schema for a simple Business Entity comprising basic attributes and relationships with people and other businesses.
• Map the JSON Schema into realistic JSON examples.

• Map the JSON Schema into a PlantUML diagram for easy visualisation by other stakeholders
• Wrap the schemas and examples up into a complete API spec ready for publication and review.

I got about 60% of the way there in a 60 minute ChatGPT session and learned a bit about how this might work in the near future.

Overall, ChatGPT was like a very dumb (and obsequious) junior developer. It seems to know the technicalities and syntax. It does a very impressive job, but makes mistakes along the way, needs to be told the obvious and sometimes drops details that were achieved earlier.

But who am I kidding - this is a computer, modelling data structures and writing code that actually compiles. Not long ago this was unthinkable magic!

Some of the highlights of our session (for me):

• It knew the format for Australian Business Number but didn't use that format until I asked
• It (correctly) argued with me over the merits of Class vs Entity-Relationship diagrams 
• When I corrected it's use of a 201 response, it knew exactly what I meant

The key thing to watch for is that ChatGPT can very confidently generate rubbish, so you still need to know what you are doing and validate the results at every step.

The lesson I learned early in this experiment, was to adjust the interaction style. ChatGPT is a conversation and so my requests to it would be like prodding it with a stick.

"Do this."

"Now do that!"

"Oh, you forgot the other thing."

Quite time consuming and frustrating.

I made much better progress when I went into more of a coding loop—editing Chat's output and using that as input for the next iteration. Part of the problem was having to iterate from scratch each time. 

I think this would work better in a pair-programming kind of interaction where we are both adjusting the same codebase.

The Possibilities

I can envisage the following kind of workflow:

• Create a PlantUML ER model. It's probably faster to complete the ER model yourself rather than try and get Chat to do it.

Once we have that, we are good to go because the rest is mostly just transformation which is Chat's inner nature.

• Map PlantUML to JSON Schema, saves a lot of tedium with JSON Schema syntax.
• Next, generate examples from the JSON Schema. Ditto with saving tedious effort.
• Back and forth iteration between variations on schema and examples to exercise the different parts of a state model.
• Now you can generate an API spec with path definitions. Chat already knows the basic CRUD methods.
• I think you'll have to assemble the final spec yourself….Chat seemed to get a bit lazy at this point with long output and we were running into token limits.

Some things I didn't try was to include a resource state model (which would help with better path generation) or to try and include API design standards and linking. They would add whole new challenges, but would add a lot of value.

I don't think you can yet do this kind of collaboration with Copilot or similar code completion tools - they are  more targeted toward in-situ code completion. Nor can you productively do it with ChatGPT because that is more conversational rather than collaborative. With the right workflow process in a custom IDE plugin, I can see how AI could really help boost productivity for API design and to ensure alignment with an enterprise API standard.

The Bigger Picture

I can definitely see how AI will make us all 10x engineers but that is a short-term advantage. AI is going to change the game, and at some point software engineering will be totally different to how we currently think and build solutions. Most of our current engineering tasks, solutions and frameworks will disappear - replaced by new ways of thinking and new ways of working with AI. Productively generating three-tier application code will be pointless because the traditional three-tier application architecture will likely disappear.

I recently watched the documentary AlphaGo - The Movie about how DeepMind beat the all-time world champion Go player Lee Sedol. If you haven't seen it, drop everything and watch it now.  It's a surprisingly emotional documentary, perhaps because it captures in microcosm what the near future likely holds for us all.

First, hubris; disbelief that a mere program could ever beat a human. Then, profound shock and existential fear as the unthinkable happens. The final message of the film is that working with AI can open up new ways of thinking and new ways of playing (working). It would be nice if that holds true.

I think the outcome will depend on whether we view AI as a competitor or a collaborator, on how we think of ourselves as human and on how we choose to share the benefits that result.