Solar Roof

I had some solar panels installed on my roof, so here’s a post with graphs & numbers & stuff!

TL;DR: should more or less cover my electricity usage; easy setup; more involved paperwork; cost about 5k€ (should get 30% of that back someday due to government incentives); expected ROI 10 years; panels themselves are less than half of the cost today.

Setup and Paperwork

Sometime this winter, I had a small array of 12 solar panels (about 20 square meters; 3.12kWp, “kWp” is maximum power rating in ideal conditions) setup on my roof. Besides the panels, a power inverter has to be set up (panels generate direct current). I had it slapped on a wall in my garage.

The setup was fairly impressive - basically one dude did it all by himself in six hours. Including getting the panels onto the roof. Efficient!

And then came the paperwork… That took about three months at Vogon pace; every little piece of paper had to be signed in triplicate, sent in, sent back, queried, lost, found, subjected to public inquiry, lost again, and finally buried in soft peat for three months and recycled as firelighters. Ok it’s not that bad; most of the paper busywork was done by the company that is doing the solar setup (I just had to sign an occasional document). But still, the whole chain of “setting up solar roof requires 20 paperwork steps” means that all these execute in a completely serial fashion and take three months.

Which means I could only “turn on” the whole thing a month ago.

How does it work?

The setup I have is a two-way electricity accounting system, where whatever the panels generate I can use immediately. If I need more power (or the sun is not shining), I use more from the regular grid. If I happen to generate more than I need, the surplus goes back into the grid for “storage” that I can recoup later (during the night, or during winter etc.). I pay about 0.03€ for kWh stored that way (regular grid electricity price is about 0.11€); the grid essentially works as a battery.

A two-way meter is setup by the energy company that tracks both the power used & power given back to the grid.

This setup is fairly new here in Lithuania; they only started doing this a couple years ago. I’ve no idea how it is in other countries; you’ll have to figure that out yourself!

Show me the graphs!

On a bright sunny day, the solar production graph almost follows the expected dot(N,L) curve :)

It generated a total of 21.4 kWh on that day. On a very cloudy/rainy day, the graph looks like this though (total 7.5 kWh generated):

My own usage averages to about 9 kWh/day, so output on a very cloudy summer day does not cover it :|

Over the course of May, daily production was on average 15.4 kWh/day:

For that month, from the 478 kWh produced I have returned/loaned/stored 371 kWh to the grid, and used 188 kWh from the grid during evenings.

All this is during near-midsummer sunny month, in a fairly northern (55°N) latitude. I’ll have to see how that works out during winter :)

What’s the cost breakdown?

Total cost of setting up everything was about 5200€:

  • Solar panels 2120€ (40%)
  • Power inverter 1310€ (25%)
  • Design, paperwork, permits, all the Vogon stuff 800€ (15%)
  • Construction work & transportation 680€ (14%)
  • Roof fixture 300€ (6%)

So yeah, this means that even if future prices of the panels are expected to fall further, by itself that won’t affect the total cost that much. I would expect that future paperwork prices should fall down too, since today that process is way too involved & complexicated IMHO.

I should get about 30% of the cost back, due to some sort of government (or EU?) programs that cover that amount of solar (and I guess wind/hydro too) installations. But that will be more paperwork sometime later this year; will see.

What’s the ROI? Why do this?

Return on investment is something like 10 years, assuming roughly stable electricity prices, and if the solar modules retain 90% of their efficiency at the end of that.

Purely from financial perspective, it’s not an investment that would be worth the hassle, at this scale at least.

However, that assumes that the price I pay for electricity is the total actual cost of electricity. Which I think is not true; there’s a massive hidden cost that we all happily ignore – but is distributed to other poor people (or future ourselves) in terms of droughts, fires, floods, political instabilities, wars and so on. Solar has some hidden cost too (mining rare minerals needed for the solar modules; that is a finite resource and has long-lasting non-trivial effects on the places where mining happens), but it feels like today that’s a lesser evil than the alternative of coal/oil/gas.

So if I can do even a tiny part towards “slightly better balance”, I’m happy to do it. That’s it!


User's POV and Empathy

Recently someone at work said “hey Aras, you write great feature overview docs, what are the tips & tricks to do them” and that got me thinking… The only obvious one I have is:

Imagine what a user would want to know about {your thing}, and write it down.

I had some more detail in there, e.g. when writing a feature overview or “proposal for a thing”, write down:

  1. How do things work right now, what are current workflows of achieving this and what are the problems with it.
  2. How will this new feature solve those problems or help in other ways.
  3. Write down what a user would need to know. Including use-cases, examples and limitations.

Now, that is simple enough. But looking at a bunch of initial docs, release notes, error messages, UI labels and other user-facing things, both at Unity and elsewhere, it’s presumably not that obvious to everyone :)

Random Examples

Release Notes

Mercurial DVCS used to have release notes that were incomprehensible (to me as a user at least). For example here, take a look at hg 3.4 notes. What does “files: use ctx object to access dirstate” or “dirstate: fix order of initializing nf vs f” mean? Are these even words?!

Even git, which I think is a somewhat Actively Hostile To The User(*) version control system, had better release notes. Check out git 2.12.0 notes for example.

(*) I know, “Yeah, well, that’s just, like, your opinion, man” etc.

Thankfully, since Mercurial 3.7 they started writing “feature overview“ pages for each release, which helps a lot to point out major items in actual human language. Now I can read it and go “oh this looks sweet, I should upgrade” instead of “I know the words, but the sentences don’t mean anything. Eh, whatever!”.

Yes, this does mean that you can’t just dump a sorted list of all source control commit messages and call them release notes. Someone has to sit down and write an overview. However, your users will thank you for that!

Pull Requests

A pull request or a code review request asks other people to look at your code. They have to spend their time doing that! Time they could perhaps spend doing something else. If you can spend five minutes making their job easier, that is often very much worth it.

I used to review a lot of code in 2015/2016 (~20 PRs each week), and the “ugghhh, not this again” reaction was whenever I saw pull requests like this. In each case, that single sentence is the only thing in PR description, with no further info besides list of commits & code diff:

  • “Updates to 2D framework + all new animation window full of win”. 374 commits, 100 files changed, 8000 lines of code.
  • “Latest MT rendering work”. 64 commits, 95 files, 3000 lines of code.
  • “Multithreaded rendering refactor”. 119 commits, 79 files, 4000 lines of code.
  • “UWP Support”. 224 commits, 219 files, 7000 lines of code.
  • “Graphics jobs (preliminary)”. 114 commits, 112 files, 2000 lines of code.

Seriously. You just spent several months doing all that work, and have one sentence to describe it?!

Sometimes just a list of commit messages is enough to describe pull request contents for the reviewers (this is mostly true for PRs that are a bunch of independent fixes; with one commit per fix). If that is the case, say that in the PR description! The above list of PR examples were very much not this case though :)

What would be good PR descriptions that make reviewer’s job significantly easier? Here’s some good ones I’ve seen:

How do you make more PRs have good descriptions?

Often I would go and poke PR authors asking for the description, especially for large ones. Something like “Wow this is a big PR! Can you write up a summary of changes in the description; would make reviewing it much easier”.

Another thing we did was make our pull request tool pre-fill the PR description field:

Purpose of this PR:

[Desc of feature/change. Links to screenshots, design docs, user docs, etc. Remember reviewers may be outside your team, and not know your feature/area that should be explained more.]

Testing status:

[Explanation of what’s tested, how tested and existing or new automation tests. Can include manual testing by self and/or QA. Specify test plans. Rarely acceptable to have no testing.]

Technical risk:

[Overall product level assessment of risk of change. Need technical risk & halo effect.]

Comments to reviewers:

[Info per person for what to focus on, or historical info to understand who have previously reviewed and coverage. Help them get context.]

This simple change had a massive impact on quality of PR descriptions. Reviewers are now more happy and less grumpy!

Commit Messages and Code Comments

A while ago I was looking at some screen resolution handling code, and noticed that some sequence of operations was done in a different order on DX11 compared to DX9 or OpenGL. Started to wonder why, turns out I myself have changed it to be different… five years ago… with a commit message that says “fixed resolution switches” (100 lines of code changed).

Bad Aras, no cookie!

Five years later (of heck, even a few months later) you yourself will not remember what case exactly this was fixing. Either add that info into the commit message, or write down comments on tricky parts of code. Or both. Especially in code that is known to be hard to get right (and anything involving resolution switches on Windows is). Yes, sometimes it does make sense to write ten lines of comments for a single line of code. Future you, or your peers, or anyone who will look into this code will thank you for that.

I know that since I have had “pleasure” of maintaining my own code ten years later :) Cue “…and if you tell that to the young people today, they won’t believe you…” quote.

Any time someone might be wondering “why is this done?” about a piece of code, write it down. My own code these days often has comments like:

// All "folders" in the solution are represented as "fake projects" too;
// with a special parent GUID and instead of pointing to vcxproj files they
// just point to their own names.
var foldersParentGuid = "2150E333-8FDC-42A3-9474-1A3956D46DE8";

or

// VS filters file needs filter elements to exist for the whole folder hierarchy
// (e.g. if we have "Editor\Platform\Windows", we need to have "Editor\Platform",
// and "Editor" too). Otherwise it will silently skip grouping files into filters
// and display them at the root.
var parentFolders = allFolders.SelectMany(f => f.RecursiveParents).Where(p => p.Depth > 0).ToArray();
allFolders.UnionWith(parentFolders);

// VS seems to load projects a bit faster if file/folder lists are sorted
var sortedFolders = allFolders.OrderBy(f => f); 

If anything else, I write these for future myself.

Empathy

What all the above has in common?

Empathy.

Putting yourself into position of the reader, user, colleague, future self, etc. What do they want to know? What makes their job/task/life easier?

Mikko Mononen once showed these amazing resources:

I’m not sure what conclusion I wanted to make, so, ehhh, that’s it. Read the resources above!


How does Visual Studio pick default config/platform?

Everyone using Visual Studio is probably familiar with these dropdowns, that contain build configurations (Debug/Release is typical) and platforms (Win32/x64 is typical):

When opening a fresh new solution (.sln) file, which ones does Visual Studio pick as default? The answer is more complex than the question is :)

After you have opened a solution and picked a config, it is stored in a hidden binary file (VS2010: {solutionname}.suo, VS2015: .vs/{solutionname}/v14/.suo) that contains various user-machine-specific settings, and should not be put into version control. However what I am interested is what is the default configuration/platform when you open a solution for the first time.

Default Platform

Platform names in VS solution can be arbitrary identifiers, however platform names defined in the project files have to match an installed compiler toolchain (e.g. Win32 and x64 are toolchain names for 32 and 64 bit Windows on Intel CPUs, respectively).

Turns out, default platform is first one from an alphabetically, case insensitive, sorted list of all solution platforms.

This means if you have Win32 and x64 as the solution platforms, then 32 bit one will be the default. That probably explains why in recent VS versions (at least since 2015), the built-in project creation wizard started naming them as x86 and x64 instead – this conveniently makes x64 be default since it sorts first.

A note again, platform names in the project have to be from a predefined set; so they have to stay Win32 and x64 and so on. Even if you use VS for editing files in a makefile-type project that invokes compiler toolchains that VS does not even know about (e.g. WebGL – for that project, you still have to pick whether you want to name the platform as Win32 or x64).

Default Configuration

When you have Debug and Release configurations, VS picks Debug as the default one. What if you have more configurations, that are more complex names (e.g. we might want to have a Debug WithProfiler LumpedBuild)? Which one will be the default one?

So, a pop quiz time! If all projects in the solution end up having this set of configurations, which one will VS use by default?

Foo
Foo_Bar
Foo Bar
Alot
AlotOfBanjos
Alot Of Bees
Debug
Debug_Lumped
Debug_Baloney
DebugBaloney

You might have several guesses, and they all would make some sense:

  • Foo since it’s the first one in the solution file,
  • Alot since it’s the first one alphabetically (and hey that’s how VS chooses default platform),
  • Debug since VS probably has some built-in logic to pick “debug” first.

Of course all these guesses are wrong! Out of the list above, VS will pick Debug_Baloney as the default one. But why?!

The logic seems to be something like this (found in this stackoverflow answer, except it needed an addition for the underscore case). Out of all configurations present:

  1. Sort them (almost) alphabetically, case insensitive,
  2. But put configs that start with debug before all others,
  3. Also config that is another one with a Whatever or _Whatever added, go before it. So A_B goes before A; Debug All goes before Debug (but DebugAll goes after Debug).
  4. And now pick the first one from the list!

I do hope there is a good explanation for this, and today VS team probably can’t change it because doing so would upset three bazillion existing projects that learned to accidentally or on purpose to depend on this.

However this means that today in our code I have to write things like this:

// We want to put spaces into default config of the 1st native program;
// see ConfigurationToVSName. Default configs in each programs might be
// separate sets, so e.g. in the editor there might be "debug_lump" and
// the standalone might only have a longer one "debug_lump_il2cpp", and
// if both were space-ified VS would pick "debug lump il2cpp" as default.
var defaultSolutionConfig = nativePrograms[0].ValidConfigurations.Default;
GenerateSolutionFile(nativePrograms, solutionGuid, projectGuids, defaultSolutionConfig);
GenerateHelperScripts();
foreach (var np in nativePrograms)
{
    GenerateProjectFile(np, projectGuids, defaultSolutionConfig);
    GenerateFiltersFile(np);
}

and this:

// Visual Studio has no way to indicate which configuration should be default
// in a freshly opened solution, but uses logic along the lines of
// http://stackoverflow.com/a/41445987 to pick default one:
//
// Out of all configurations present:
// 1. Sort them (almost) alphabetically, case insensitive,
// 2. But put configs that start with "debug" before all others,
// 3. Also config that is another one with a " Whatever" or "_Whatever" added,
//    go before it. So "A_B" goes before "A"; "Debug All" goes before "Debug"
//    (but "DebugAll" goes after "Debug").
// 4. And now pick the first one from the list!
//
// Our build configs are generally underscore-separated things, e.g. "debug_lump_il2cpp".
// To make the default config be the first one, replace underscores in it with
// spaces; that will make it sort before other things (since space is before
// underscore in ascii), as long as it starts with "debug" name.
string ConfigurationToVSName(CApplicationConfig config, CApplicationConfig defaultConfig)
{
    if (config.IdentifierNoPlatform != defaultConfig.IdentifierNoPlatform)
        return config.IdentifierNoPlatform;
    return config.IdentifierNoPlatform.Replace('_', ' ');
}

This is all trivial code of course, but figuring out the logic for what VS ends up doing did took some experimentation. Oh well, now I know! And you know too, even if you never wanted to know :)

And voilà, debug_lump is picked by VS as the default in our auto-generated project files, which is what I wanted here. Without any extra logic, it was picking debug_lump_dev_il2cpp since that sorts before debug_lump as per rules above.

That’s it for now!


A case of slow Visual Studio project open times

I was working on some new code to generate Visual Studio solution/project files, and that means regenerating the files and checking them in VS a lot of times. And each time, it felt like VS takes ages to reopen them! This is with Visual Studio 2017, which presumably is super-turbo optimized in project open performance, compared to previous versions.

The VS solution I’m working on has three projects, with about 5000 source files in each, grouped into 500 folders (“filters” as VS calls them) to reflect hierarchy on disk. Typical stuff.

On my Core i7-5820K 3.3GHz machine VS2017 takes 45 seconds to open that solution first time after rebuilding it, and about 20 seconds each time I open it later.

That’s not fast at all! What is going on?!

Time for some profiling

Very much like in the “slow texture importing” blog post, I fired up windows performance recorder, and recorded everything interesting that’s going on while VS was busy opening up the solution file.

Predictably enough, VS (devenv.exe) is busy during most of the time of solution load:

Let’s dig into the heaviest call stacks during this busy period. Where will this puppy with 16k samples lead to?

First it leads us through the layers of calls, which is fairly common in software. It’s like an onion; with a lot of layers. And you cry as you peel them off :)

So that is Visual Studio, seemingly processing windows messages, into some thread dispatcher, getting into some UI background task scheduler, and into some async notifications helper. We are 20 stack frames in, and did not get anywhere so far, besides losing some profiling samples along the way. Where to next?

A-ha! Further down, it is something from JetBrains. I do have ReSharper (2016.3.2) installed in my VS… Could it be that R# being present causes the slow project load times (at least in VS2017 with R# 2016.3)? Let’s keep on digging for a bit!

One branch of heavy things under that stack frame leads into something called CVCArchy::GetCfgNames, which, I guess, is getting the build configurations available in the project or solution. Internally it’s another onion, getting into marshaling and into some immutable dictionaries and concurrent stacks.

And another call branch goes into CVCArchy::GetPlatformNames, which seemingly goes into exactly the same implementation again. ¯\_(ツ)_/¯

So it would seem that two things are going on: 1) possibly R# is querying project configurations/platforms a lot of times (once for each file?), and 2) querying that from VS is actually a fairly costly operation.

VS seemingly tries to fulfill these “what configurations do you have, matey?” queries in an asynchronous fashion, since that also causes quite some activity on other VS threads. Hey at least it’s trying to help :)

Some of which causes not much actual work being done, e.g. this thread spends 1.5k samples doing spin waits. Likely an artifact of some generic thread work system not being quite used as it was intended to, or something.

There’s another background thread activity that kicks in towards end of the “I was busy opening the project” period. That one is probably some older code, since the call stack is not deep at all, and it fairly quickly gets to actual work that it tries to do :)

Let’s try with R# disabled

Disabling R# in VS 2017 makes it open the same solution in 8 seconds (first time) and 4 seconds (subsequent opens). So that is pretty much five times faster.

Does this sound like something that should be fixed in R#, somehow? That’s my guess too, so here’s a bug report I filed. Fingers crossed it will be fixed soon! They already responded on the bug report, so things are looking good.

(Edit: looks like this will be fixed in R# 2017.1, nice!)

Visual Studio 2015 does not seem to be affected; opening the same solution with R# enabled is about 8 seconds as well. So this could be Microsoft’s bug too, or an unintended consequence of some implementation change (e.g. “we made config queries async now”).

Complex software is complex, yo.

While at it: dotTrace profiler

Upon suggestion from JetBrains folks, I did a dotTrace capture of VS activity while it’s opening the solution. Turns out, it’s a pretty sweet C# profiler! It also pointed out to basically the same things, but has C# symbols in the callstacks, and a nice thread view and so on. Sweet!

So there. Profiling stuff is useful, and can answer questions like “why is this slow?”. Other news at eleven!


Developer Tooling, a week in

So I switched job role from graphics to developer tooling / build engineering about 10 days ago. You won’t believe what happened next! Click to find out!

Quitting Graphics

I wrote about the change right before GDC on purpose - wanted to see reactions from people I know. Most of them were along what I expected, going around “build systems? why?!” theme (my answer: between “why not” and ¯\_(ツ)_/¯). I went to the gathering of rendering people one evening, and the “what are you doing here, you’re no longer graphics” joke that everyone was doing was funny at first, but I gotta say it to you guys: hearing it 40 times over is not that exciting.

At work, I left all the graphics related Slack channels (a lot of them), and wow the sense of freedom feels good. I think the number of Slack messages I do per week should go down from a thousand to a hundred or so; big improvement (for me, anyway).

A pleasant surprise: me doing that and stopping to answer the questions, doing graphics related code reviews and writing graphics code did not set the world on fire! Which means that my “importance” in that area was totally imaginary, both in my & in some other people’s heads. Awesome! Maybe some years ago that would have bothered me, but I think I’m past the need of wanting to “feel important”.

Not being important is liberating. Highly recommended!

Though I have stopped doing graphics related work at work, I am still kinda following various graphics research & advances happening in the world overall.

Developer Tooling

The “developer tools” team that I joined is six people today, and the “mission” is various internal tools that the rest of R&D uses. Mostly the code build system, but also parts of version control, systems for handing 3rd party software packages, various helper tools (e.g. Slack bots), and random other things (e.g. “upgrade from VS2010 to VS2015/2017” that is happening as we speak).

So far I’ve been in the build system land. Some of the things I noticed:

  • Wow it’s super easy to save hundreds of milliseconds from build time. This is not a big deal for a clean build (if it takes 20 minutes, for example), but for an incremental build add 100s of milliseconds enough times and we’re talking some real “developer flow” improvements. Nice!
  • Turns out, a lot of things are outdated or obsolete in the build scripts or the dependency graph. Here, we are generating some config headers for the web player deployment (but we have dropped web player a long time ago). There, we are always building this small little tool, that turns out is not used by anything whatsoever. Over here, tons of build graph setup done for platforms we no longer support. Or this often changing auto-generated header file, is included into way too many source files. And so on and so forth.
  • There’s plenty of little annoyances that everyone has about the build process or IDE integrations. None of them are blocking anyone, and very often do not get fixed. However I think they add up, and that leads to developers being much less happy than they could be.
  • Having an actual, statically typed, language for the build scripts is really nice. Which brings me to the next point…

C#

Our build scripts today are written in C#. At this very moment, it’s this strange beast we call “JamSharp” (primarily work of @lucasmeijer). It is JamPlus, but with an embedded .NET runtime (Mono), and so the build scripts and rules are written in C# instead of the not-very-pleasant Jam language.

Once the dependency graph is constructed, today it is still executed by Jam itself, but we are in the process of replacing it with our own, C# based build graph execution engine.

Anyway. C# is really nice!

I was supposed to kinda know this already, but I only occasionally dabbled in C# before, with most of my work being in C++.

In a week I’ve learned these things:

  • JetBrains Rider is a really nice IDE, especially on a Mac where the likes of VisualStudio + Resharper do not exist.
  • Most of C# 6 additions are not rocket surgery, but make things so much nicer. Auto-properties, expression bodies on properties, “using static”, string interpolation are all under “syntax sugar” category, but each of them makes things just a little bit nicer. Small “quality of life” improvements is what I like a lot.
  • Perhaps this is a sign of me getting old, but e.g. if I look at new features added to C++ versions, my reaction to most of them is “okay this probably makes sense, but also makes my head spin. such. complexity.“. Whereas with C# 6 (and 7 too), almost all of them are “oh, sweet!”.

So how are things?

One week in, pretty good! Got a very vague grasp of the area & the problem. Learned a few neat things about C#. Did already land two pull requests to mainline (small improvements and warning fixes), with another improvements batch waiting for code reviews. Spent two days in Copenhagen discussing/planning next few months of work and talking to people.

Is very nice!