Your Tech Blog Isn't Helping

For the third time in a week, I found myself having to look at source code to solve a simple problem. First it had been Terraform, then it was the AWS SDK, and now it was Terraform again. Each time I'd fallen into a gap in the documentation, but not one so big it ought not to be solved with one or two simple web searches.

Except for one small problem.

I was having the kind of trouble you'd expect someone reasonably familiar with Go, AWS and Terraform might have if they tried to surface a Go lambda as a websocket API through API Gateway for the first time. Needing an attribute that exists but isn't in the Terraform documentation, putting the wrong type of event as a parameter for my handler, not realising that even a mock $connect route appears to still need an integration response once it has a custom authoriser connected to it, that sort of relatively entry-level self-inflicted failure you have when you know exactly what you need but have no idea which particular casting of the runes is necessary to make it reality.

The complication here is a search along the lines of "API Gateway Terraform custom authoriser" or perhaps even something more targeted like "golang APIGatewayCustomAuthorizerRequest AuthorizationToken empty " happens to overlap perfectly with some rather unhelpful search spaces.

The first is usually the documentation I've just been reading, because it will be the biggest and often most-visited aggregation of my search terms. Great if I was not the kind of person who already has the documentation open in three separate tabs by this point, and is down at the depth where the AWS SDK docs start emulating classic MSDN's tendency toward, "Method FrobWidgets: frobs widgets. Parameters: widgets - an array of widgets" without any further enlightenment forthcoming on why one might encounter an "application exception: cannot frob" error when calling this method.

The second overlapping search space is basic entry-level questions like "how do I get the ARN of a function I want to use in" or "how do I pass token in HTTP request?"where people paste in their code and it happens to be using the same resources and types that I'm trying to get information on. I know the title mentions blogs but really the complaint is about user-generated content as a whole, and this is where I start to get going. For something that was the great hope of 2008, the state of StackOverflow in 2021 is painful, centred around two main problems:

• The "too similar: somebody has already asked a question about Python, while this relates to a completely different situation we shouldn't have two questions about the same language" of early StackOverflow has gone too far the other way and for any common, simple problem there will be as many questions as people who've had the problem, swamping any hope of finding a rarer and more complex problem unless you get really lucky with a unique search term.
• StackOverflow answers from about 2014 onward are awful. Questions used to be answered by people who knew their tools and generously took the time to explain why their answer worked. Answers now are more like, "I had the same problem of 'computer won't turn on' - I tried walking in a circle three times, removing and replacing my Funko Pops on the desk, opening the window, plugging the computer in, and reducing the height of my chair. I've decided the chair thing must have been the solution so now I'm going to type 'lower your chair three inches' with no context." This will be followed by several comments from people who also lowered their chair, noticed while they were down there that the computer was unplugged, but reply, "chair thing worked for me as well".

Hold that thought, because I'm about to get to the main event.

The third, and by far the largest overlapping search space is an endless sea of tech blogs, an unbroken stretch of "Setting up a websocket API with AWS Lambda and API Gateway" from here to the point at which you get bored clicking through pages of search results.

The total number of these things we need is at most 1. Maybe 1 per implementation language, if we're going to be charitable. Because they all follow exactly the same format:

• 19 paragraphs about what a computer is, why enterprises are turning to AWS, how Terraform is such an amazing tool for automation and IaC and DevOps to make sure the page is bursting with as many SEO-friendly terms as possible.
• A whole page full of callouts and diagrams on how to install Terraform, as a standing monument to how complicated you can make the instruction "put the executable in the place"
• A partial extract of a Terraform configuration file, presented without context.
• Some diagrams of AWS infrastructure.
• A mysterious skip in which the author seems to get to the point at which they'd have the same problem you have, then emerge the other side without explaining how or why they got there.
• A link to a GitHub repo, the code in which is broken.

If you're desperate enough to follow all of this and fix the obviously broken bits of the repo, you will find that not only does it have exactly the same problem you have, it also accepts any non-empty string as a valid authorisation token and most of the resources created are world-writable with a note of "# IAM policies for illustration only, be sure to secure appropriately in production!"

This is why I tend not to blog a new technical HOWTO every other week. For a start, I have a painful awareness that anything which I find exciting enough to spend a few hours documenting and editing will also be new to me and I'll be telling people how to make noob mistakes that I'll never go back and edit once I know better. This is why I explicitly disclaimed my Go series as written by a beginner and made it more about how to approach a new programming language than anything else. (It also has a practical purpose, in that it's what we use to help existing developers onboard to Go without needing to go through 12 steps of "this is how you hello world", "this is how you add two numbers together", "strongly typed means that...")

The second is the world does not need yet another "how to make a sort of broken API Gateway setup with a couple of basic errors and inappropriate permissions" article. It already has 127,648 of the things. I give no value adding the 127,649th. I doubt anyone needs "Matt Kimber shows you how to build a footgun with API Gateway" and even if they did, they can just read one of those 127,648 existing ones and imagine it being 37% angrier and more cynical with the occasional reference to 1960s pop no-one's heard of thrown in. If anything, I would worsen the problem which is that there probably is a well-founded article on the subject out there, but it's in blue text on a cyan background with links to several highly opinionated rants about UNIX and is therefore getting buried by your slick Ghost blog where you spent 100 times as long optimising SEO as you did testing whether your code actually worked.

(Again, that Go series? When I wrote it I couldn't find anything along the lines of "what this looks like for an existing programmer who's a beginner in the language". Everything out there was, "how I, a Go expert of some considerable 3 days, will teach you to write a 'Hello World' which not only breaks every established Go convention but also every once in a while outputs 17,000 repetitions of the character 'd' because I didn't test it properly.")

But why? Why do we do this?  Why do we write StackOverflow answers and blogs about things we don't understand, turning every quest for information into a trawl through mediocrity in the hope of unearthing the one thing written by someone who knew what they were doing, the only thing that actually needed to be written?

Well, I guess you could put it down to some combination of:

• Stupid hiring pipeline checklists where you can't just be a good programmer, you must have a blog and a GitHub and a StackOverflow profile to be exceptional and stand out, no matter how broken or trivial the contents. (I keep threatening to friends that one of these days I'm going to subvert this and replace an interview round by taking someone's broken blog/code/answer and saying, "right - I took this from your online profile, if you and I can get it working exactly as advertised within the hour without changing any of the material as published you've got the job". I've not been quite that mean yet.)
• Similarly, stupid internal promotion checklists that talk about being a member of the community but don't bother to follow up with whether those contributions were actually useful to said community or merely hindered it.
• The secret hope that someone at a tech giant (but, like, not one of the evil ones) will discover your writing and immediately decide that such a tortured genius should be given, interview-free, a £1m/year post with equity for writing such elegantly crafted outreach articles.
• The simple narcissistic joy of being not just a programmer, but a programmer with a tech blog.
• OK, I admit it, the writing part is actually quite fun.
• As a means to vent their gathering frustration and disillusionment that we're rapidly sliding back toward the programming environment of the 1990s in which the only resources were books, your own ingenuity and occasionally a bit of source code if you were lucky.

I haven't really got a point here other than maybe "don't". Well, maybe the standard advice to creators; don't go chasing what's hot and popular, because by the time you get there it's not and it isn't. Find your own voice and your own things you're interested in writing about, and do the things no-one else is doing. I think this may have been what I was getting at with Improvement and imperfection, if you wonder why I have both a "do write tech blogs" and "don't write tech blogs" article. That way the odds are good of having at least a small loyal following and while it may feel like screaming into a void at least the void isn't dully echoing back a thousand cries of, "but that doesn't even WORK".