Go

After reading the technicalwriting.dev post on embeddings I thought to myself - this seems like something that I can implement quite quickly and would serve as a good starting point for some hands on experience with machine learning. I want to start small, so something as simple as filling the related posts section of this blog seems like an ideal candidate to get my hands dirty. This blog is made with Hugo and uses the related content feature which provides a list of related posts based on the tags & keywords you use. While I have no quarrels with the mechanism, I thought this would be a good place to try and experiment with embeddings. ...

About cache stampedes I often end up in situations where I need to cache this or that. Often, these values are cached for a period of time. You’re probably familiar with the pattern. You try to get a value from cache, if you succeed, you return it to the caller and call it a day. If the value is not there, you fetch it(most likely from the database) or compute it and the put it in the cache. In most cases, this works great. However, if the key you’re using for your cache entry gets accessed frequently and the operation to compute the data takes a while you’ll end up in a situation where multiple parallel requests will simultaneously get a cache miss. All of these requests will independently load the from source and store the value in cache. This results in wasted resources and can even lead to a denial of service. ...

SQLC has become my go-to tool for interacting with databases in Go. It gives you full control over your queries since you end up writing SQL yourself. It then generates models and type safe code to interact with those queries. I won’t go over the basics here, if you feel like it you can try their interactive playground. Dynamic queries Frequently I end up needing to filter the data by a set of fields in the database. This set of fields is often determined by the caller, be it via REST API or other means. This means that the code I’m writing has to support dynamic queries, where we query by a subset of fields. ...

I’ve seen many discussions about whether Go should add enum support to the language. I’m not going to bother arguing for or against but instead show how to make due with what we have in the language now. A very short enum intro Enumerated types, or enums, represent finite sets of named values. They are usually introduced to signal that variables can only take one of the predefined values for the enum. For example, we could have an enum called Colors with members Red, Green, Blue. Usually, the members are represented as integer values, starting from zero. In this case, Red would correspond to 0, Green to 1 and Blue to 2 with Red, Green and Blue being the names of corresponding members. They help simplify the code as they are self-documenting and explicitly list all possible values for the given type. In many languages enums will also return compile errors if you’ll try to assign to an invalid value. However, since enums do not exist in Go, we do not have such guarantees. ...

Go comes with great tools for profiling out of the box. It’s one of the features I’ve come to love. I’m not going to go into detail about pprof, but if you need a primer, Julia Evans has a great post about it on her blog. Instead, I’ll try and show a real world example of the optimizations you can make using pprof. I’ll use pprof on mouthful, a gin based server, to see what I could do to make it just a bit faster. It turns out I can detect a rather stupid mistake I’ve made. ...

There are quite a few components to this blog now. There’s an AWS S3 bucket the blog gets served from, there’s a backend service responsible for the mouthful comments, there’s another instance of the service for the mouthful demo page, with another S3 bucket to serve the page for it. There’s also a yet unused service living in the same “cluster” as these two(It’ll get added to this blog soon™). There’s quite a few private github repositories that in unison with a Jenkins instance make this all work. Oh, and everything is running under Docker, with ECS handling all the orchestration for that. For the data stores, I use dynamodb, mysql and sqlite. There’s also some miscellaneous bits and bobs, like the grafana instance that plots response times and uptimes of the APIs. I’ve tried to keep as much as I can inside AWS but due to cost constraints I had to do some workarounds. So how does it all work then? ...

When writing Go code, I often end up with lots of enums that I use to fork my logic in a switch statement. Take this enum: type MyEnum int const ( One MyEnum = iota Two MyEnum = iota Three MyEnum = iota Four MyEnum = iota Five MyEnum = iota )} I’ll then end up with a switch in a part of my code, like so switch myEnum { case One: err := DoSomeOtherStuff() if err != nil { return err } case Two: err := DoSomeMagicalStuff() if err != nil { return err } case Three: err := DoSomeExoticStuff() if err != nil { return err } case Four: err := DoSomeOtherStuff() if err != nil { return err } case Five: err := DoSomeStuff() if err != nil { return err } }} ...

When I created mouthful, I was intending it to be rather light and not feature rich but after getting a few feature requests getting in, I’ve decided to expand it. One of the issues was a request to reuse logon credentials for the admin panel. For that, I’ve needed OAuth. I did not have much prior experience with OAuth, so it did intimidate me a bit. However, after implementing OAuth for mouthful, I can say that nowadays - it’s rather easy including OAuth in your applications as well. It’s also a rather good idea to do so as people behind the providers such as github or facebook are probably going to do a better job than a lone developer like me will at securing your credentials. Anyway, with this post I’d like to show how easy it is to add OAuth to your gin project. ...

I haven’t been blogging much lately. That’s due to the fact that most of my spare time went to creating mouthful - a commenting server that I’ve since switched this blog to. Before that I was using isso. Here’s why I did it. The issue with isso While I really like isso, there’s one problem that kills it for me. Under no load, the backend consumes nearly 50MB of memory on my server. You might say that 50MB is nothing nowadays but I disagree. I’m running everything under AWS t2.nano instances, meaning I only have 500MB of memory to play with. Since I’m running them as docker hosts for my ECS cluster with a few applications on them memory is an expensive commodity. Since I’ve become quite fond of GO, I’ve also looked at commento but at the time of writing it did not have moderation support. I really wanted that. I knew that I could squeeze in the functionality I needed an only use a few MB of RAM. Currently, the commenting service runs with most of the functionality enabled with a stable memory usage of under 7MB. Take this with a grain of salt though, as I do not have many comments or visitors currently. ...

<rant> I absolutely despise OpenApi(well, swagger. I’ll call it swagger since it was that for most of my career). It’s supposed to make the process of creating and documenting an API easier. But does it really?… Here’s what I think about it. Writing the swagger schema is so damn tedious From my experience when someone mentions swagger, you’ll get dragged into design first - code later mindset rather quickly. While it sounds good on paper the practical implications of writing a swagger schema just make me want to curl in a ball in the corner of the office. The swagger schema is so explicit and such a pain to write. We’ve got what I’d call a rather simplistic API by corporate standards at my work. I invite you to “design” it first. The swagger definition for it is 19808 lines long. I’m not kidding. Just think about it. 20 fucking thousand lines of JSON. Good luck designing that. Now, some of you will say “but wait a moment, why not generate the swagger schema from code”. While that does sound like a great idea on paper I really fail to see the point of it. If I document my code properly I can already generate documentation that is not going to be as tedious or as explicit. If the code is undocumented, swagger does not provide any sort of a benefit there. ...