So it’s been six months since I switched to doing build engineering / developer tooling. Here are some random things I learned.

It’s a service team

We are working on a live codebase, with I don’t even know how many (~500?) developers on it, and it has to keep on building and not crumble down with additional code / people / stuff. This means the team I’m on is a “service / live ops” team to a large extent, does a bunch of support and short term problem solving, in addition to steering the whole codebase build towards a hopefully better future.

That is not a surprise; I fully expected that.

What was a surprise: I thought it would be much worse to be on a “service” team.

A build system is one of these areas where “if it works well, no one notices it”, and yes, in many cases when people talk about it they talk about problems they are having. And so before starting all this I had mentally prepared myself to jump into a super thankless job, where everyone else is complaining about state of things and shouting at you for all the issues there, etc.

So far it’s less thankless, and way less people shouting at me than I expected!

It is true that when people hop onto our Slack channel, they are coming with some problem they are having. They might provide excellent information and articulate what is happening in good detail, or they might be very vague and confused; they might be just stating the problem or expressing frustration, disappointment, anger or whatever else. However, all that is basically complaining about computers, or software, or codebase; I don’t remember a single time where anyone did a “stupid build people” type of complaints. For some reason I expected some amount of issues to reach an ad-hominem level… maybe I’ve read too many bad stories on the internet, and none of them turned out to be applicable here. Great!

That said, an occasional “yo! just wanted to say that you are all doing a good job, keep it up!” is appreciated, when working in a service-like team. Go and tell that right now to people making sure your wireless works, your servers are building stuff, your office fruits are delivered on time, your patches are predictably shipped to the cloud, …

When someone has a problem, they want it solved

When someone has a problem and could not solve it for themselves in X amount of time, they hopefully turn for help. Be it coworkers, Slack, email, twitter or whatever. The composition of my team right now is that people working on “build system” are fairly remote, so the way people ask us is mostly Slack messages.

Here’s the main thing: for their problem, what they are mostly interested in is “make it disappear”. The most important thing is to unblock them, so that they can continue doing whatever they were trying to do.

First figure out what is going on; do the minimal thing to make the problem disappear for them (“graft this changeset”, “change file X line Y to Z”, “pass this argument on command line”, “delete this folder from build artifacts and continue build”, “click that setting in your IDE”, …) - whatever quick fix/hack it takes. Add a TODO for yourself to fix it properly, and actually follow through on that.

If it’s an issue that happened more than once or twice, add to a list of “known issues” (e.g. via pinned messages in Slack channel), with good, searchable wording and clear instructions how to workaround/solve it. Keep the known issues list more or less up to date!

What people are not interested in, when they come to your support channel:

1. A lecture on how Well Actually It’s Not An Issue You See, You Are Doing Things Wrong. Sure they might be doing something wrong; they don’t need a patronizing lecture though. Short, to the point information on what should be done instead. Also take a note - maybe they do have a valid point? Maybe the commands / options / file locations / whatever is indeed confusing and error prone? Maybe it could or should be improved, either in code changes or at least in better error messages / documentation / training?

2. Whose fault the issue is (though more on that separately below).

3. Why the issue happens, and an explanation of some innards of build code. Some people might be interested in details on why the issue happens; but first solve the issue for them, and provide details as an optional bonus information, if needed.

4. A long rant about how your team is understaffed and can’t deal with that many issues. Every team would want to do more things than they have resources for; and yes maybe your team is in a more dire situation – that’s not what the inquiry is about however. Solve their problem first. If you feel like it, maybe mention that “yeah we’ll solve this eventually, but so far didn’t have time yet”. But better talk with someone who can fix the staffing situation (management, etc.).

Own your f***ups

There’s one exception I have for the “no one is interested in whose fault the issue is” guideline – if it’s your fault, then saying “Sorry about this! My fault, fixing ASAP” is acceptable :)

Over half a year of working on build stuff, I think I have made a dozen or so embarrassing things that caused people frustration & wasted their time. It’s not ideal, in retrospect for at least half of them “wot were you thinking aras” is a good summary. I did try my best to provide workarounds, apologize and fix as soon as possible. And then hope that I did more than a dozen things that improved the build experience.

And again, if it’s a problem caused by anyone else, there’s no need to point fingers. You can still say “sorry! we’ll be fixing ASAP” of course.

Build performance / stability is an uphill battle

Codebases tend to grow over time – more stuff being added, more developers doing it; and usually “old things” can not be retired and removed as fast. This means that even to keep “build times” constant over time, someone has to actively spend time chipping away on various inefficiencies.

Over last month I’ve spent some time optimizing full/clean builds (one example), and for some build targets it’s effectively “twice as fast” now. However, those slow parts did not exist half a year ago! So over that time period, the full build time has not improved… it’s “just” no longer twice as bad.

Similar with build stability - various bugs in the build process & dependency graph are not some old problems; and instead they are caused by new systems landing, new ways that code is split up, new platforms, and so on. Just maintaining same level of build sanity is actual work that needs to be done, and it’s never finished. On the plus side… hey, job security! :)

Job satisfaction?

Kinda related to “this was supposed to be a thankless job” (turns out, not so much), I’ve been thinking on what about it can be considered “job satisfaction”. For me, it’s perhaps the “I feel like I’m being useful” part. That’s a very broad statement, however it also means that I can pick many areas to work in, compared to something like “the only way to make me happy is if I get to write GCN assembly all day long” :)

The work I’m doing is not some sort of cutting edge research or “interesting” problems. There’s no best paper awards for finding out how to split up .vcxproj and .csproj files for most optimal Visual Studio intellisense experience, for example. But at this point in my life, I’m perfectly fine with that!

Do I still get to delete code?

Happy to report – yes! I won’t reach my extrapolated goal of removing 200kloc this year though. So far this year: 82kloc added & 143kloc removed (so made the codebase 61kloc smaller). However first two months were still doing graphics stuff; since I started doing build engineering the stats are 50kloc added, 87kloc removed (net effect 37kloc smaller). Not bad!

Well, that’s it. I’ll get back to working on some C# project generation stuffs.

A couple months ago I added profiling output to our build system (a fork of JamPlus). It’s a simple Chrome Tracing view, and adding support for that to Jam was fairly easy. Jam being written in C, I found a simple C library to do it (minitrace) and with a couple compilation fixes it was working and had profiler blocks around whatever tasks the build system was executing.

That by itself is not remarkable. However… once you have profiling output, you can start to…

Notice Things

The other day I was doing something unrelated, and for some reason looked at a profiler output of Unity editor build I’ve just done. “Common sense” about C++ codebases tells us that usually it’s the linking time that’s The Worst, however here it was not that:

There’s a whole gap right before linking starts, when everything else is done, just the build is waiting for one C++ file to compile. I was busy with something else right then, so added a TODO card to our team task board, to be looked at later.

Sometime later I was doing a different build (of just the “engine” without the editor bits), and looked at the profiler too:

Now that looks pretty bad. The total build time (this was a “full rebuild”) was ten minutes, and almost five of them were “wait for that one C++ file to finish compiling”. The file in question had a compile time of seven minutes.

This sounded like a build inefficiency that is too large to ignore, so I dug in for a bit.

Aside: at the moment our small “build team” has not even started to look into “full build” performance, for various reasons. Mostly repaying tech debt (with interest!), optimizing incremental build performance and improving things like IDE projects. Optimizing full build time we will get to, but haven’t just yet.

7 minutes compile time for one file? Craaazyy!

Yes, it is crazy. Average compile time for C++ files in this project and this config (win64 Release) is about 2 seconds. 400+ seconds is definitely an outlier (though there’s a handful of other files that take 30+ seconds to compile). What is going on?!

I did some experiments and figured out that:

• Our Lump/Batch/Unity builds are not the culprit; there’s one .cpp file in that particular lump that takes all the time to compile.
• MSVC compiler exhibits this slow compilation behavior; clang compile times on that file are way better (10+ times faster).
• Only Release builds are this slow (or more specifically, builds where inlining is on).
• The fairly old VS2010 compiler that we use is not the culprit; compiling with VS2015 is actually a tiny bit slower :(
• In general the files that have 30+ second compile time in Release build with MSVC compiler tend to be heavy users of our “SIMD math library”, which provides a very nice HLSL-like syntax for math code (including swizzles & stuff). However the implementation of that is… shall we say… quite template & macro heavy.
• That 7 minute compile file had a big SIMD math heavy function, that was templated to expand into a number of instantiations, to get rid of runtime per-item branches. This is in CPU performance sensitive code, so that approach does make sense.

Whether the design of our math library is a good trade-off or not, is a story for another day. Looks like it has some good things (convenient HLSL-like usage, good SIMD codegen), and some not so good things (crazy complex implementation, crazy compile times on MSVC at least). Worth discussing and doing something about it in the future!

Meanwhile… could some lo-tech changes be made, to speed up the builds?

Speeding it up

One easy change that is entirely in “build systems” area even, is to stop including these “slow to compile” files into lump/batch/unity builds. Lump builds primarily save time on invoking the compiler executable, and doing preprocessing of typically similar/common included code. However, all that is sub-second times; if a single source file takes 10+ seconds to compile then there’s little point in lumping. By itself this does not gain much, however for incremental builds people won’t have to wait too long if they are working on a fast-to-build file that would have ended in the same lump as a slow file.

Could also somehow make the build system schedule slow files for compilation as soon as possible. The earlier they start, the sooner they will finish. Ideally the build system would perhaps have historical data of past compile times, and use that to guide build task scheduling. We don’t have that in our build system (yet?)… However in our current build code, moving slow files out of lumps makes them be built earlier as a by-product. Good!

The above things didn’t actually speed up compilation of that 7-minute file, but were trivial to do and gained a minute or so out of full build time (which was 10 minutes originally).

And then I tried something that I didn’t have high hopes of working - in that slow file, factor out parts of the big templated function into smaller, less-templated functions.

Fairly trivial stuff; if you’re into fancy refactoring IDEs that is done by “select some code lines, press Extract Method/Function button”. Except this is C++, and the IDEs don’t quite get this right (at least in this code), so you do it manually.

I split off five or so functions like that and… compile time of that file went from 420 seconds to 70 seconds. That is six times faster!

Of course, factoring out functions can mean they are no longer inlined, and the original code will have to call them. So that’s time spent in passing the arguments, doing the jump etc. However, it could make parent function use registers better (or worse…), could result in lower amount of total code (less I$ misses), etc. We have profiled the code change on several platforms, and the difference in performance seems to be negligible. So in this particular case, it’s a go!

Now of course, a minute to compile a C++ file is still crazy, but additional speedups probably involve changing the design of our math library or somesuch – needs more work than someone clueless randomly trying to speed it up :)

The build profile looks much better after the change. No longer gaps of “all the CPU cores wait for one job to finish” type of stuff! Well ok, linking is serial but that is old news. Full build time went from 10 minutes down to 5min 10sec, or almost 2x faster!

Takeaways

• Add some sort of profiling data capture to everything you care about. Had I not added the profiler output, who knows when (or if?) someone would have noticed that we have one super-slow to compile C++ source file. In my case, adding this simple profiling capture to our build system took about a day… most of that spent learning about Jam’s codebase (my first change in it).
  • Seriously, just do it. It’s easy, fun and worth it!
• Every template instantiation creates additional work for the compiler’s optimizer; pretty much as if the instantiated functions were literally copy-pasted. If a templated function was slow to compile and you instantiate the template 20 times… well now it’s very slow to compile.
  • Factoring out parts into less-templated functions might help things.
• Complicated template-heavy libraries can be slow to compile (“well duh”, right). But not necessarily due to parsing (“it’s all header files”); the optimizer can spend ages in them too. In MSVC case in this code, from what I can tell it ends up spending all the time deciding what & how to inline.
• Speeding up builds is good work, or at least I like to think so myself :) Less time spent by developers waiting for them; better throughput and lower latency in build farms, etc. etc.

This June we’ve spent two weeks traveling in South Korea, so here’s a writeup and a bunch of photos.

Caveats: my first trip there, and very likely I mis-planned something or missed some obvious “oh my god can’t miss that” things to see or do.

Our trip was two weeks; with myself, my wife and two kids (14 and 8yo). If you’re going alone, or for a honeymoon, or with a group of friends, the experience might be very different.

Planning

We did the itinerary ourselves, using the same old method of wikipedia + wikitravel + random blogs + asking around. Planning via writing rough outline in google docs and marking up places in a custom google map.

And here was the first challenge: Google does not have that much information on Korea, especially in the maps department. Apparently this is due to an old law that prevents mapping/GIS data to be exported for foreign countries & companies (see this quora answer). Does not make much sense these days, but oh well.

It’s not a big problem, but made me realize how much I have learned to depend on Google for planning trips. “A lot”, and that’s slightly scary.

What we came up with was this: in two weeks, we go to Seoul (3 nights), Sokcho (2 nights), Gyeongju (1 night), Busan (3 nights), Jeju (2 nights), Seoul (2 nights). Did not have any concrete goals, except “see some cities, nature, people, temples, food”. Wanted to check out some K-pop concert, but about the only one happening during that time (of Monsta-X) got sold out in like two minutes :|

Getting Around

We did not rent a car, instead were traveling mostly by metro & bus. T-money card for local transit works conveniently; metro goes often & on time. Both Seoul and Busan and really large cities, so getting from one place to another takes a lot of time. Outside of big cities, figuring out how exactly to get from A to B, is a bit of a challenge. For example bus terminal in Sokcho barely has any information in English, and not many people speak it either.

As mentioned before, Google Maps does not have exhaustive information, so I used a bunch of other apps to figure out buses, trains, places to go and so on. Some of them are Korean only, but after some fiddling I was able to semi-randomly get by.

Speaking of language… my older kid has learned to read & write Hangul (and some Korean words) as a hobby, and that was useful! The writing looks intimidating for someone who sees it the first time, but actually it’s fairly simple! There’s only a few dozen letters; they are just arranged in a non-linear fashion (instead each syllable is arranged into a square block).

I do recommend to check it out. If nothing else, know that ㅇ at top is silent (does not do anything), and at the bottom is “ng”. With just that you can already tell some words apart!

One place where we should have rented a car is Jeju. It seems small, but public transport there is… not terribly good, shall I say. Rare, slow and crowded buses, and fairly long walking distances from bus stops to anything. Do rent a car or hire a fulltime taxi!

Staying

Wow, Korean apartments are small! It makes sense since South Korea is one of the most densely populated countries. We mostly used airbnb, and learned the hard way that many listings with “two rooms” are actually one area with two beds tucked to the ceiling and a narrow staircase to get into them :)

Stayed a couple nights in a hanok in Bukchon, and that was nice!

Aside about airbnb: lately overall it feels kinda “meh”. A number of years ago airbnb meant you could experience places that are somehow personal; places where someone lives. These days, more and more airbnb apartments are rented full-time, with the same darn Ikea stuff in all of them. Often you don’t even meet anyone, just get a message with they keypad code and that’s it. It’s kinda like going to a hotel, without any of the advantages of a hotel. Meh!

Culture

We learned that selfies are important, especially if you’re showing V sign or some sort of heart symbol in them. Overall they are pretty serious about that stuff:

So we tried to blend in,

Cult of “youthful, perfect” beauty is fairly big here, it seems. At least it’s kinda equally divided among sexes. Maybe even larger part of ads & posters with “here’s someone beautiful” image had a guy in them. Makeup is important, clothes are important, etc. I’ve read somewhere that “if you go with just t-shirt and jeans, people will think you’ve given up on life”. We went mostly in t-shirts and jeans :)

Feels like Koreans love to write up instructions everywhere, especially about what you’re not supposed to do. Can’t stray from the predetermined path. Can’t take photos in that direction. etc. etc.

We were surprised at the amount of meat, specifically pork, that is everywhere. Being a vegetarian or vegan must be pretty hard there. Another surprise - lack of bread (something hard to imagine for an Eastern European). Food is generally rice, pork, seafood and kimchi, and fairly spicy.

Photo Impressions

Almost all photos are taken by my wife. Equipment: Canon EOS 70D with Canon 24-70mm f/2.8 L II and Sigma 8-16mm f/4.5-5.6. Some taken by iPhone 6 or iPhone SE.

Seoul, Samcheong-Dong 삼청동

The area where we stayed in a hanok village, close the old imperial palaces. Narrow, steep streets. Mountains in the background on one side, highrise glass buildings on the other side. And churros that are better than a boyfriend!

Seoul, Bugaksan mountain 북악산

The hiking trail goes along the old city wall, right north to the Blue House (residence of the president). Probably for that reason you need to fill out registration form, there are guards every hundred meters, and a few layers of barbed wire, cameras and stuff. I got used to not having all that back home!

Seoul, Cheonggyecheon stream 청계천

Very nice walk in the center of Seoul. Great contrast of the nature & surrounding highrise.

Sokcho 속초

Went to Sokcho for a hike in Seoraksan national park nearby. After Seoul, this felt small and a bit empty? That said, after Seoul a lot of places can feel that way :) Apparently a lot of people go to Sokcho for the seafood, but none of us are big seafood fans, so that part was out. We walked around the seaside and went to the park the next day.

Seoraksan National Park 설악산국립공원

Sinheungsa temple (신흥사) is right at the entrance to the park.

We walked for couple hours in Ulsanbawi (울산바위) direction. Did not get to the actual rock formation since it’s a longer hike, and would probably be too hard for the kids.

Took a cable car towards Gwongeumseong fortress (설악산 권금성), around 800 meters elevation.

The photos above make it look as if there’s not that many people in the park. That was definitely not the case though. It was Saturday, and it was packed with visitors; apparently Koreans love walking or hiking in their national parks. We were very impressed by 2-4 year old kids walking and climbing stones on fairly long trails.

Gyeongju 경주

Used to be capital of the ancient Silla kingdom. A lot of tumuli and other olden things around.

There’s a big Bulguksa 불국사 temple nearby. Overall temples in Korea - at least the ones we’ve seen - feel more like museum sites than active temples. Very different impression from the buddhist temples that we saw in China, for example. South Korea is a fairly non-religious country, by the way.

Busan 부산

We liked Busan a lot! However the city is very spread out so getting from one place to another took a lot of time on the metro. We stayed in Haeundae area based on some internet advice.

The apartment we stayed in was impressively tiny! Which is fairly typical for the whole Korea, I guess.

Busan, Haedong Yonggungsa temple 해동 용궁사

An hour bus ride from Busan, a nice temple on the seaside.

Busan, Gamcheon Village 감천문화마을

OMG the streets are steep there! This was lovely, just take a local bus there instead of climbing all the way up from metro stop like we did :)

Jeju Island 제주도

Took a flight to Jeju from Busan Gimhae airport, and we stayed on the southern part.

Jeju is often listed as a “must visit” type of destination, but frankly we were not terribly impressed. Maybe because we made a mistake of thinking that we can get by public transportation like elsewhere in Korea – do not do that; just rent a car or hire a fulltime taxi! Oh right, I already mentioned this above… Overall, the whole place felt a bit of “too touristy” for our tastes at least.

The volcanic ash/rock formations - Yongmeori Coast (용머리해안) - near Sanbangsan Mountain (산방산) were pretty cool! There’s a Sanbanggulsa Temple (산방굴사) right there too.

Seoul, Itaewon 이태원

Flew back to Seoul Gimpo airport, and our last stay was in Itaewon district. Generally in Korea you see very few foreigners; not so in Itaewon! We had nice food, went to the N Tower and the Leeum museum. Was nice!

And then the next morning it was a long flight home!

I had some solar panels installed on my roof, so here’s a post with graphs & numbers & stuff!

TL;DR: should more or less cover my electricity usage; easy setup; more involved paperwork; cost about 5k€ (should get 30% of that back someday due to government incentives); expected ROI 10 years; panels themselves are less than half of the cost today.

Setup and Paperwork

Sometime this winter, I had a small array of 12 solar panels (about 20 square meters; 3.12kWp, “kWp” is maximum power rating in ideal conditions) setup on my roof. Besides the panels, a power inverter has to be set up (panels generate direct current). I had it slapped on a wall in my garage.

The setup was fairly impressive - basically one dude did it all by himself in six hours. Including getting the panels onto the roof. Efficient!

And then came the paperwork… That took about three months at Vogon pace; every little piece of paper had to be signed in triplicate, sent in, sent back, queried, lost, found, subjected to public inquiry, lost again, and finally buried in soft peat for three months and recycled as firelighters. Ok it’s not that bad; most of the paper busywork was done by the company that is doing the solar setup (I just had to sign an occasional document). But still, the whole chain of “setting up solar roof requires 20 paperwork steps” means that all these execute in a completely serial fashion and take three months.

Which means I could only “turn on” the whole thing a month ago.

How does it work?

The setup I have is a two-way electricity accounting system, where whatever the panels generate I can use immediately. If I need more power (or the sun is not shining), I use more from the regular grid. If I happen to generate more than I need, the surplus goes back into the grid for “storage” that I can recoup later (during the night, or during winter etc.). I pay about 0.03€ for kWh stored that way (regular grid electricity price is about 0.11€); the grid essentially works as a battery.

A two-way meter is setup by the energy company that tracks both the power used & power given back to the grid.

This setup is fairly new here in Lithuania; they only started doing this a couple years ago. I’ve no idea how it is in other countries; you’ll have to figure that out yourself!

Show me the graphs!

On a bright sunny day, the solar production graph almost follows the expected dot(N,L) curve :)

It generated a total of 21.4 kWh on that day. On a very cloudy/rainy day, the graph looks like this though (total 7.5 kWh generated):

My own usage averages to about 9 kWh/day, so output on a very cloudy summer day does not cover it :|

Over the course of May, daily production was on average 15.4 kWh/day:

For that month, from the 478 kWh produced I have returned/loaned/stored 371 kWh to the grid, and used 188 kWh from the grid during evenings.

All this is during near-midsummer sunny month, in a fairly northern (55°N) latitude. I’ll have to see how that works out during winter :)

What’s the cost breakdown?

Total cost of setting up everything was about 5200€:

• Solar panels 2120€ (40%)
• Power inverter 1310€ (25%)
• Design, paperwork, permits, all the Vogon stuff 800€ (15%)
• Construction work & transportation 680€ (14%)
• Roof fixture 300€ (6%)

So yeah, this means that even if future prices of the panels are expected to fall further, by itself that won’t affect the total cost that much. I would expect that future paperwork prices should fall down too, since today that process is way too involved & complexicated IMHO.

I should get about 30% of the cost back, due to some sort of government (or EU?) programs that cover that amount of solar (and I guess wind/hydro too) installations. But that will be more paperwork sometime later this year; will see.

What’s the ROI? Why do this?

Return on investment is something like 10 years, assuming roughly stable electricity prices, and if the solar modules retain 90% of their efficiency at the end of that.

Purely from financial perspective, it’s not an investment that would be worth the hassle, at this scale at least.

However, that assumes that the price I pay for electricity is the total actual cost of electricity. Which I think is not true; there’s a massive hidden cost that we all happily ignore – but is distributed to other poor people (or future ourselves) in terms of droughts, fires, floods, political instabilities, wars and so on. Solar has some hidden cost too (mining rare minerals needed for the solar modules; that is a finite resource and has long-lasting non-trivial effects on the places where mining happens), but it feels like today that’s a lesser evil than the alternative of coal/oil/gas.

So if I can do even a tiny part towards “slightly better balance”, I’m happy to do it. That’s it!

Recently someone at work said “hey Aras, you write great feature overview docs, what are the tips & tricks to do them” and that got me thinking… The only obvious one I have is:

Imagine what a user would want to know about {your thing}, and write it down.

I had some more detail in there, e.g. when writing a feature overview or “proposal for a thing”, write down:

1. How do things work right now, what are current workflows of achieving this and what are the problems with it.
2. How will this new feature solve those problems or help in other ways.
3. Write down what a user would need to know. Including use-cases, examples and limitations.

Now, that is simple enough. But looking at a bunch of initial docs, release notes, error messages, UI labels and other user-facing things, both at Unity and elsewhere, it’s presumably not that obvious to everyone :)

Random Examples

Release Notes

Mercurial DVCS used to have release notes that were incomprehensible (to me as a user at least). For example here, take a look at hg 3.4 notes. What does “files: use ctx object to access dirstate” or “dirstate: fix order of initializing nf vs f” mean? Are these even words?!

Even git, which I think is a somewhat Actively Hostile To The User^(*) version control system, had better release notes. Check out git 2.12.0 notes for example.

(*) I know, “Yeah, well, that’s just, like, your opinion, man” etc.

Thankfully, since Mercurial 3.7 they started writing “feature overview” pages for each release, which helps a lot to point out major items in actual human language. Now I can read it and go “oh this looks sweet, I should upgrade” instead of “I know the words, but the sentences don’t mean anything. Eh, whatever!”.

Yes, this does mean that you can’t just dump a sorted list of all source control commit messages and call them release notes. Someone has to sit down and write an overview. However, your users will thank you for that!

Pull Requests

A pull request or a code review request asks other people to look at your code. They have to spend their time doing that! Time they could perhaps spend doing something else. If you can spend five minutes making their job easier, that is often very much worth it.

I used to review a lot of code in 2015/2016 (~20 PRs each week), and the “ugghhh, not this again” reaction was whenever I saw pull requests like this. In each case, that single sentence is the only thing in PR description, with no further info besides list of commits & code diff:

• “Updates to 2D framework + all new animation window full of win”. 374 commits, 100 files changed, 8000 lines of code.
• “Latest MT rendering work”. 64 commits, 95 files, 3000 lines of code.
• “Multithreaded rendering refactor”. 119 commits, 79 files, 4000 lines of code.
• “UWP Support”. 224 commits, 219 files, 7000 lines of code.
• “Graphics jobs (preliminary)”. 114 commits, 112 files, 2000 lines of code.

Seriously. You just spent several months doing all that work, and have one sentence to describe it?!

Sometimes just a list of commit messages is enough to describe pull request contents for the reviewers (this is mostly true for PRs that are a bunch of independent fixes; with one commit per fix). If that is the case, say that in the PR description! The above list of PR examples were very much not this case though :)

What would be good PR descriptions that make reviewer’s job significantly easier? Here’s some good ones I’ve seen:

How do you make more PRs have good descriptions?

Often I would go and poke PR authors asking for the description, especially for large ones. Something like “Wow this is a big PR! Can you write up a summary of changes in the description; would make reviewing it much easier”.

Another thing we did was make our pull request tool pre-fill the PR description field:

Purpose of this PR:

[Desc of feature/change. Links to screenshots, design docs, user docs, etc. Remember reviewers may be outside your team, and not know your feature/area that should be explained more.]

Testing status:

[Explanation of what’s tested, how tested and existing or new automation tests. Can include manual testing by self and/or QA. Specify test plans. Rarely acceptable to have no testing.]

Technical risk:

[Overall product level assessment of risk of change. Need technical risk & halo effect.]

Comments to reviewers:

[Info per person for what to focus on, or historical info to understand who have previously reviewed and coverage. Help them get context.]

This simple change had a massive impact on quality of PR descriptions. Reviewers are now more happy and less grumpy!

Commit Messages and Code Comments

A while ago I was looking at some screen resolution handling code, and noticed that some sequence of operations was done in a different order on DX11 compared to DX9 or OpenGL. Started to wonder why, turns out I myself have changed it to be different… five years ago… with a commit message that says “fixed resolution switches” (100 lines of code changed).

Bad Aras, no cookie!

Five years later (of heck, even a few months later) you yourself will not remember what case exactly this was fixing. Either add that info into the commit message, or write down comments on tricky parts of code. Or both. Especially in code that is known to be hard to get right (and anything involving resolution switches on Windows is). Yes, sometimes it does make sense to write ten lines of comments for a single line of code. Future you, or your peers, or anyone who will look into this code will thank you for that.

I know that since I have had “pleasure” of maintaining my own code ten years later :) Cue “…and if you tell that to the young people today, they won’t believe you…” quote.

Any time someone might be wondering “why is this done?” about a piece of code, write it down. My own code these days often has comments like:

// All "folders" in the solution are represented as "fake projects" too;
// with a special parent GUID and instead of pointing to vcxproj files they
// just point to their own names.
var foldersParentGuid = "2150E333-8FDC-42A3-9474-1A3956D46DE8";

or

// VS filters file needs filter elements to exist for the whole folder hierarchy
// (e.g. if we have "Editor\Platform\Windows", we need to have "Editor\Platform",
// and "Editor" too). Otherwise it will silently skip grouping files into filters
// and display them at the root.
var parentFolders = allFolders.SelectMany(f => f.RecursiveParents).Where(p => p.Depth > 0).ToArray();
allFolders.UnionWith(parentFolders);

// VS seems to load projects a bit faster if file/folder lists are sorted
var sortedFolders = allFolders.OrderBy(f => f); 

If anything else, I write these for future myself.

Empathy

What all the above has in common?

Empathy.

Putting yourself into position of the reader, user, colleague, future self, etc. What do they want to know? What makes their job/task/life easier?

Mikko Mononen once showed these amazing resources:

• The Paradox of Empathy by Scott Jenson.
• Gel 2009 talk by magician Jamy Ian Swiss.

I’m not sure what conclusion I wanted to make, so, ehhh, that’s it. Read the resources above!