gamescapades

reviews

foh@gamescapad.es (gamescapades) — Tue, 01 Jul 2025 08:14:43 +0200

Mirror's Edge Catalyst is a Crap Game

May 19, '19

home

foh@gamescapad.es (gamescapades) — Tue, 01 Jul 2025 08:14:43 +0200

essays

foh@gamescapad.es (gamescapades) — Tue, 01 Jul 2025 08:14:43 +0200

Increasing Diversity Decreases Team Blind Spots

Nov 22, '22
Planet Ghibli Part 3 - How AI can increase inequality in society if we do not engage in diversity

Aug 14, '20

contact

foh@gamescapad.es (gamescapades) — Tue, 01 Jul 2025 08:14:43 +0200

Want to get in touch? Great! Email is probably the safest bet if you want a response this century.

email: h.siljebrat@gold.ac.uk
linkedin: Henrik Siljebråt
twitter: foh
github: fohria

about

foh@gamescapad.es (gamescapades) — Tue, 01 Jul 2025 08:14:43 +0200

gamescapades is the ~~personal blog~~ public rant storage of Henrik Siljebråt.

gamescapades is a blog about life,
games
and my lovely adventures in both.
because life is a game.
a horrible and wonderful game.
and then you die.
maybe respawn is enabled?

To know more you can visit the website named after myself

If you're lazy, this is me in hash tag form:
#neuroscience #videogames #ledzeppelin

Here are some selected reviews of this site:

"not totally horrendous"

-- zoë o'shea

I used 11ty to create this site, using a modified Hitchens theme. I've also made the effort to not rely on any third party sites for fonts and scripts. That means your adblocker should show nothing is blocked. I believe that such efforts are good for the health of the interwebs, for security and privacy reasons.

The current post/redesign ratio is: 2 (10 posts, 5 redesigns)

Increasing Diversity Decreases Team Blind Spots

foh@gamescapad.es (gamescapades) — Tue, 22 Nov 2022 01:00:00 +0100

I was at an event the other day. Three AI talks followed by a panel discussion with the speakers.

All three had these headsets with the mic itself at the end of an arm you can swing up and down. One of the speakers/panelists had long hair. Every time they turned their head to look at a fellow panelist, their hair bumped into the microphone so it swung upwards. Every time, they had to adjust the mic back down, next to their mouth. This was quite funny once I noticed it but also, why did this happen?

a) the headset was broken/worn out
b) the headset was designed and tested exclusively by people with short hair

For the sake of argument, I assume it’s B^[1], turning the anecdote into a great example of a team having blind spots in their knowledge. The headset designers simply didn’t think of what may happen if you have long hair and it bumps into the mic. If they had at least one person on the team with long hair, they would've discovered the issue.

This has hugely important implications for AI.

Many of the questions to the panel had to do with ethics; transparency, trust, explainability, correctness, how to avoid discrimination and bias. My impression was the panelists have good intentions, as many answers brushed into diversity but it was never explicitly stated. What concerned me was the leaning towards technologist approaches.

One of the answers to the question how to avoid biased datasets and discriminatory results/predictions was "that's the job of the data scientist". But even if you’re the best data scientist in the world, due to your particular life experience and limited knowledge, there will be aspects you miss. You will have blind spots.

To decrease the unknown number of such blind spots, you have to increase diversity. Important to understand here is that diversity is not just about race, creed, gender, socioeconomic factors and other common diversity metrics. Those just mentioned are obviously important, but diversity is also about having more perspectives than that of computer science and maths. Like historians or sociologists. Ideally as integral parts of the team. Because bias and oppression is a societal problem, and AI systems use data from and have consequences in society.

If you are not aware of this, you risk having a situation where you have an AI division made up and led by people who think everything is a technical problem. Facebook/Meta's AI division is the typical example here, which is led by Yann LeCun who quit twitter in 2020 over failing to see the bigger picture. More recently, Facebook/Meta released and then pulled an AI system advertised as being able to generate scientific papers and wiki articles, with LeCun commenting that "it's no longer possible to have some fun". The system was advertised as competent, not fun. In other words, LeCun continues to be unaware of the responsibility he and others at his company has to be mindful of the consequences of AI systems.

AI already impacts society and will have even deeper impact in the future. If you are serious about using these systems to make society better - for everyone - then please take diversity to heart.

You can do so by, for example, reading my Planet Ghibli series. But don't take my word for it. If you're in the AI field and haven't watched Ruha Benjamin's ICLR 2020 keynote you should do yourself, your customers and society a service: watch it.

I realise the irony of basing my discussion on a biased assumption, but please bear with me :) ↩︎

Holistic Sustainability - Automating subtitles for Swedish public service with OpenAI Whisper

foh@gamescapad.es (gamescapades) — Mon, 26 Sep 2022 02:00:00 +0200

Holistic Sustainability is my personal term^[1] to describe that sustainability is not only about the environment. For a truly sustainable society, we should minimise all forms of inequality, because they are all interconnected. If we cannot respect each other, how are we ever to respect the environment we are all part of and the earth we all share?

Immigration and integration are two very loaded terms these days, and were big topics in the recent Swedish election. Without going into detail on these complex questions, part of the discussion revolves around difficulties to get new citizens feel included. Despite the fact that Swedes in general can and do speak English, most of Swedish society requires one to know Swedish. Personally, I believe in providing people with the tools they need to succeed. So what if we could provide people a way to learn Swedish language and culture simultaneously?

"How is AI shifting power?" -- Pratyusha Kalluri

SVT is Sweden's public service TV. They provide a streaming service called SVT Play with news, movies, documentaries, game shows - you name it. Most of their videos have subtitles in Swedish, but no other language. Many of their videos (like the one below) even have forced Swedish subtitles. Such subtitles are of course great if you know Swedish and have trouble hearing. But what if you don't know Swedish?

I've learned a lot of English by watching English language movies and TV shows with subtitles in Swedish. By being exposed to a language like that you get used to it and start to pick up phrases. Wouldn't it be a great idea if SVT had, at the very least, English subtitles?

This is where OpenAI Whisper comes in. Whisper is a "general-purpose speech recognition model", which means that it's a computer program that can transcribe voice into text. It can transcribe voices from 98 different languages, and - get this - even translate the text into English.

I spent part of my weekend playing with it, and it is surprisingly easy to use. There are already many tutorials online on setup and basic use of Whisper, so I shall leave technical details for another post. The only relevant technical information I should make clear is that I used the "medium" size model. Results would likely be better with the "large" model. Yet, Whisper works surprisingly well.

The result

If you cannot see the video below, check it out at Vimeo. I put the English subtitles at the top as the video already had hard coded Swedish subtitles at the bottom.

If you know Swedish, you'll see the translation is quite good. Remember, this is with the medium size model, so presumably the large model would be even better. At around the 1min50s mark, the subtitle output gets ahead of itself and is a sentence ahead of what is being said for the rest of the video. We could adjust this manually, or there could be a Whisper setting to make this better from the start. Overall though, the subtitles do what subtitles should!

Also, since Whisper translates audio, we don't get a translation of the title text at the start of the video. But if our primary audience is people learning Swedish, this is not a top priority anyway.

Thoughts and next steps

I think the result is really impressive, especially considering this was my first test. I haven't looked into all the Whisper settings, or had a chance to test the large model.

However, Whisper is not capable of real-time translation and to get decent output speed you need a fairly beefy computer. But the system works well enough that it would be possible to create a web site where the user enters a URL to, for example, SVT Play, and after a few minutes you're provided with a subtitled video. Such a service is the obvious next step for this project, so keep an eye out for my next blog post by following me on linkedin, twitter or subscribe to the rss feed of this blog.

Oh, by the way, I'm looking for a job in the ML/AI space, hire me!

PS. If you like this idea and can fund cloud compute for this project, please get in touch.

Most likely, what I mean by the term is already included in existing theories like for example intersectionality. I just finished a PhD and have not had time to read up on these topics properly. I'm obviously not the first to come up with this specific term, seeing as holisticsustainability.com was registered in 2009 ↩︎

Planet Ghibli Part 3 - How AI can increase inequality in society if we do not engage in diversity

foh@gamescapad.es (gamescapades) — Fri, 14 Aug 2020 02:00:00 +0200

Welcome to part 3 of Planet Ghibli! This time we're getting political.

As we have seen in Part 1 and Part 2, machine learning (or "AI" in mainstream media) is time- and resource intensive. This because first of all you need data, which may or may not take up much of your time, because maybe you use an existing dataset. Another is picking an algorithm, where it may not be obvious which one is most suitable for your use case. Then, depending on your intent there is an unknown amount of time taken to train your model (run all the data through the algorithm), evaluate the result, and perhaps try again a bunch of times before you're happy with the output. As we saw in the Part 2 discussion, this phase can be tricky because outputs may vary enormously depending on the methods used.

With complicated use cases, like playing video games or generating text, we need huge amounts of data and huge architectures (algorithms). This quickly becomes so expensive that only huge companies are able to afford to train such systems. Take for example the text-generation system GPT-3 developed by OpenAI. It can do some amazing things like generate code from text and create novel solutions. Without going into mathematical details, it's one of the largest machine learning systems ever released and used what one article states is the fifth most powerful supercomputer in the world, with hundreds of thousands of processors and a cost of at least $1 billion. I'm not sure how much trial and error was needed, probably some, but also much less than I personally would need because these researchers are experts in their field after all.

If you consider the amount of work we put into getting nice looking pictures for our Planet Ghibli style transfer, and the interaction between input and output; that was fairly time consuming right? And it was time consuming despite us having access to resources in the form of a graphics card. So with time and money becoming prohibitively expensive for individuals or even small organisations; what do you think will be the effect on society if we as regular citizens cannot investigate these systems ourselves? Is it a good thing we have to trust huge companies that they've made these systems fair?

In their paper on GPT-3, the authors do discuss how their language model reflects biases in society, how it can be misused and the energy costs of training versus use when training is done. I want to say they should be lauded for including this, but at the same time such discussions should be the default. Nevertheless, it's great they have that discussion as it shows they've thought about these issues and are (hopefully) open to further discussion about them. What about when the company is not transparent?

If we have no possibility of investigating for ourselves the interaction between data, algorithms and their combined judgments; how can we trust automated systems that decide who will get jobs, loans, medical treatments and court sentences? The short answer is: we absolutely cannot.

Why shouldn't we trust automated decision making?

"deep learning: computational depth without historical or sociological depth is superficial learning"
-- Ruha Benjamin

Because these systems are trained on existing data, they can only reinforce existing stereotypes. For example a system trained on predicting future crime is futile. Since such a system only relies on past data, that past data is usually biased towards police already being more aggressive and making more arrests in poor areas and towards minorities. So a predictive system will continue to predict that rich white people do not commit crime. And I hope you agree with me that black lives matter.

Compare the just mentioned example of crime prediction to the different outputs we got when doing style transfer in the previous parts of this blog series. We can only change the output by changing the data we use as input, and in some cases entire groups of elephants disappeared from the output. As consumers of a pre-made system, we only see the final output so without insight we will never know what data is being considered and how it's used.

Traditionally, if you applied for a bank loan, there was a person you could ask about why you can't get one. No matter what we may think of an economic system that builds on debt, knowing why you can't get that loan means you'll know what to do before trying again. With an opaque black box there is no why. And such systems are increasingly being used for job applications, bank loans and insurance.

The danger here is that if we do not stop and think, we run the risk of these systems further reinforcing existing biases and stereotypes. Just a few examples are how facial recognition can't distinguish between people of color, facebook shows job ads in a discriminatory fashion, any system that uses binary gender data is transphobic and prediction of child abuse punishes poor families. A huge issue here, and incidentally the same reason why self driving cars still don't work, is that most often these systems can't reliably catch outliers or handle situations they have never seen before. In other words; minority groups run the risk of disappearing as the data is averaged.

Shifting the power

Pratyusha Kalluri gives us this insightful thought:

"It is not uncommon now for AI experts to ask whether an AI is ‘fair’ and ‘for good’. But ‘fair’ and ‘good’ are infinitely spacious words that any AI system can be squeezed into. The question to pose is a deeper one: how is AI shifting power?"

In most of the discussed cases, I would like to believe the consequences were not intended by the researchers and engineers. It's rather a result of software engineering teams not being diverse, existing datasets not being diverse and that software engineers are trained in maths and programming but not in ethics, diversity and societal impact. So this discussion needs to be taken seriously. And what's more concerning here is that highly influential companies and people in the machine learning world seem to not fully understand this discussion and/or grasp their own role.

Such as when it was found in 2015 that a Google algorithm tagged photos of black people as gorillas. Two and a half years later, their solution was to remove gorillas from the possible tags. With all the data Google/Alphabet has available and the resources to gather, together with their hundreds or even thousands of brilliant employees with PhDs in computer science; was this really the best solution they could find? Or is it a matter of not caring to allocate resources to increase equality?

Another more recent example is how a system trained to depixelate faces had the effect of turning Barack Obama into a white man. Yann Lecun - receiver of the Turing award for his work in deep learning and currently Vice President and Chief AI Scientist at Facebook - replied that this was a consequence of the data set used. Use another data set and the problem goes away. He was criticised by Timnit Gebru for this statement being overly simplified because the problem is bigger than that; it's about why this sort of thing keeps happening and why did the authors not even think of trying their system with a diverse set of skin colours? Lecun refused to see the bigger picture and kept defending his position from a technical view. He later quit Twitter and posted a long statement on facebook about his personal values of freedom and justice.

To connect this drama to our experimentation with style transfer, it's a little like if we had stopped testing with the Princess Mononoke input picture and concluded that we would get a different result with a different input picture. It's technically correct, but fails to consider what our purpose is and how our methods of data collection and algorithm selection feeds into that purpose.

Furthermore, unless you're a researcher it may be difficult to evaluate if the algorithm works for the dataset you're using. Perhaps the data you have would need algorithmic tweaks that the researchers never thought of testing for. But developers use off-the-shelf algorithms all the time, they don't have the knowledge and/or time to make their own.

The authors of the paper in question have since added a section about biases, and some would perhaps criticise them for doing that only after discussion happened. But to be fair, that's how science should work; issues are pointed out and through discussion we move forward. This is where Lecun's answer becomes problematic in my view. He's technically correct about the data set being an issue, and you and I can see how he's correct about that, using our our style transfer as a visual simile. But Lecun seems incapable of looking beyond mathematics, defending his statement instead of being curious, and feeling so hurt for being called out that he completely quits Twitter. When the head of AI for Facebook cannot handle a discussion about issues of bias in AI/ML and saying it's only about data, it's obvious he doesn't understand the larger issue. Not to mention it says something about the company culture within Facebook.

The sad consequence for everyone is that the outlook of using AI to shift power - to empower the disadvantaged - instead of reinforcing existing inequalities, is so much bleaker when the world's most influential companies act indifferent.

"Most people are forced to live inside someone else's imagination"
-- Ruha Benjamin

So what can we do?

As software developers, ML/AI practitioners and data scientists, it's essential to gain a broader perspective than mathematical formulas. We have to be able to put the artefacts we create into a broader sociological and historical context. So learning more about those contexts is one action. Another action is to hire diverse teams. Not just diverse based on race or gender but also of perspective. Hire historians and sociologists. It doesn't matter they won't understand the details of your algorithms, you can explain the concepts so they can be valuable contributors anyway. And if you happen to not be great at explaining things, hire someone who can! A good educator can also help you be transparent by explaining to your customers how your product works.

As citizens of the world, we have an obligation to not accept the status quo. Human rights is not a done deal, we have to keep pushing for better transparency, inclusiveness and justice. For ourselves, for our fellow humans, for the life we share this planet with and for Mother Earth herself. A very important part of that is to listen. Listen to the people saying they are mistreated. Understand where they are coming from. Only then can we understand how our own actions may impact the greater picture of systemic inequalities.

My own understanding of these issues is only superficial at this point. Thanks to #BLM, XR, Laurie Penny, Timnit Gebru's recommendations and many others I've started to see how many different kinds of inequality are connected. But my to-read list^[1] is still way longer than my already-read-list and there's much I do not know.

Among the amazing people I've found there's one person in particular I want to promote. Ruha Benjamin gave a brilliant keynote at ICLR this summer. I really, really, really recommend you watch her keynote. And the Q&A.

So I will end this ~~rant~~ essay with yet another quote from Benjamin, one I find particularly inspiring:

"Remember to imagine and craft the worlds you cannot live without, just as you dismantle the ones you cannot live within."

This list is obviously non-exhaustive:
Angela Saini - Superior
Cathy O'Neil - Weapons of Math Destruction
Ruha Benjamin - Race After Technology
Safiya Umoja Noble - Algorithms of Oppression
Virginia Eubanks - Automating Inequality
Shoshanna Zuboff - The Age of Surveillance Capitalism ↩︎

Planet Ghibli Part 2 - Making a Video

foh@gamescapad.es (gamescapades) — Fri, 31 Jul 2020 02:00:00 +0200

Welcome to the second part of our Planet Ghibli adventure. In the first part, we styled single images and now we shall attempt to style an entire video. The video we will style is this trailer for Planet Earth II.

So how do we do this? Well, the simple and straight forward answer is; we extract all the frames from the video and do the style transfer on each frame.

Quick links for this post (a.k.a. TLDR)

Code repository

Final result: Styled Trailer

Discussion

Before we move on

Remember from Part 1 it took us 1-2 minutes for each image? The trailer is 2 minutes and 46 seconds long, so if we have 24 frames every second... Yeah. That's almost 4000 images, and thusly around 3 days. For 128 pixel resolution images.

Time to go look for more efficient approaches! One of the fanciest is using something like CycleGAN. But from what I understand looking into it, the computational requirements are high. What might work better is fast neural style transfer (original paper).

This only takes a few seconds per image, if we have a trained model. So what does it mean to have a trained model?

Training models briefly explained

In part 1, our pytorch code used a pre-trained object recognition network to calculate values for images. Someone had already gone through the process of "teaching" this network to recognise objects in images, by "showing" it thousands and thousands of images and pointing out where the objects are^[1]. By exploiting aspects of this pre-trained object recognition network, we could run our style and input images through it, calculate difference values, and manipulate an output image so that it became a mix between the style and input.

The fast neural transfer also uses a pre-trained object recognition network, but additionally, it trains a "image transformation network" (see Figure 2 in the paper for a nice overview). By running many images through this system, this image transformation network learns to output a styled image without having to calculate any differences between the style, input and output. It just changes the input image into an output image directly.

In other words, we could say that when we train our fast neural transfer, it's similar to what happens in part 1, but instead of treating each successive image as a new and separate case, the image transformation network will first learn how to style the first image, then it learns how to style the first and second image, then the first, second and third... you get the picture. Hah! Picture. Pun not intended, but pun happened.

The downside here is we need thousands of images to train on. So it will take some time to do this training for our chosen style image, but when it's done, each frame of our video will only take seconds. So in a way, we're betting here that the training will take less time than the 3 days or so we estimate to use the method from part 1 to style the entire video.

I should mention that when I make this bet, I take into consideration that I've access to a GTX980Ti graphics card at university. It accelerates training significantly compared to my laptop, on which this kind of deep learning is basically pointless. We will discuss this at the end of this post.

Actually training our model(s)

I cloned the git repo mentioned earlier for fast neural style, and used their instructions to train a model using the same style image we used in Part 1. I used the image of Sir David and our custom Planet Ghibli "logo" as test images, as seen here:

Style image to the far left and our test images to be styled. Click for larger versions.

One training run on the COCO2014 image dataset (the one recommended in their readme), took around 3h using a GTX980Ti for CUDA acceleration. Good thing I've access to such a card at the university, or it would likely not have been worth it compared to those 3 days we spoke of in the beginning of this post. Deep learning without GPU acceleration, like on my laptop (and most other non gaming laptops), is basically impossible unless it's for small toy examples^[2].

The first training run used the default parameter settings for content-weight and style-weight; 1e5 and 1e10, respectively, and gives us these results:

Images styled with style-weight=1e10. Click for larger versions.

These look okay, but to my eyes more like color and blur filters or something than actual style. They don't look "painting-like" if you get what I mean? Let's try using style-weight=1e11 instead!

Wait, wait; I hear you. What is this style weight business? Again referring back to Part 1, the style weight tells the training algorithm to give more weight to the style than the content. By adjusting both the style-weight and content-weight parameters we can fine tune the look of our output images. How do we find good values, you ask? Well, my friend, that's the secret about these things. You have to test and retest. Luckily for us, the readme tells us their example images have used values between 1e10 and 1e11 for the style weight.

So! Another three hours go by and we have a new model to test with our images:

Images styled with style-weight=1e11. Click for larger versions.

Notice those red stripes on Sir David's shirt? And in the water in the bottom right of the elephant shot? You recognise it? If not, scroll back up to the style image from Nausicaa that we use. Then continue to the next paragraph.

You're right, those are the windmill sails! How did they get there? Mayhaps you figured it out already, but remember we just spoke of what style weight does? By increasing the weight - the importance - of the style image over the content images during training, more of the style "seeps through". In our case, it seems that during training, one of the features identified by the object recognition network, was these sails. Again, looking at the Nausicaa image, those sails do stand out from the rest of the picture so it's not surprising it's a stand out feature.

Either way, this isn't what we want right? Sure, both the test images now look more "painted" but we don't want those sails. Let's try using style-weight=5e10, a value in-between the first and second attempts. Another three hours of training and now we get this:

Images styled with style-weight=5e10. Click for larger versions.

These aren't too bad! To be honest I slightly prefer the results of the method in Part 1, but these are good enough. We have already spent nine hours training, plus a few extra hours fiddling around with the code.

Extracting frames and running the style transfer

I downloaded the trailer using youtube-dl, which if you didn't know can be used to download video clips from twitter, reddit and other sites as well. Very useful when you want to save those cute animal gifs for future use.

Next step is to use another wonderfully useful little application; ffmpeg^[3]. It can convert between video formats quickly and do many other nifty things with video and audio. For our use case, it can extract all the frames from our trailer into individual jpg files at 24 frames per second:

ffmpeg -i input.mp4 -r 24/1 out%03d.jpg

Then we extract the audio:

ffmpeg -i input.mp4 -q:a 0 -map a audio.mp4

What we get out is almost 4000 images (!), and it's impressive, at least to me, that this process only takes a few minutes even on my crappy laptop.

Style transfer

Even though the fast style transfer is much quicker than the original one, each image still takes up to 10 seconds on my laptop. That would be 11h for 4000 images. So again, I use the office graphics card, which takes only 12 minutes in total. That's because our image dataset easily fits into the memory of the graphics card, and pytorch intelligently identifies this and transfers all the images to the graphics memory, converts them, and then sends the images back to the CPU and saves to disk.

Also, the code we're using has no support to do many images in batch, so I created a wrapper file, which I uploaded to my forked repository on GitHub.

Putting the pieces back together

With all the images styled and downloaded to my laptop, we use ffmpeg to put the pieces back together:

ffmpeg -r 24 -i out%03d.jpg -i audio.mp4 -c:v libx264 -c:a mp3 -r 24 -shortest output.mp4

The above command assumes you have not changed the filenames from the extracted ones :)

The final result can be seen on Vimeo

Discussion

If you check out the vimeo video, you'll see that some scenes don't look that great. Others look really cool! So like in the previous post, we see this interaction between input and output where we can't be sure of how things turn out until we try it.

When we trained our model we used an existing dataset of thousands of images, training our image transformation network to output styled images in a way that's an average of all those thousands of pictures. So it could be the case we would get better results if we either trained on extracted images from the trailer. Another way to improve the result would be to have separate transformation networks for different parts of the trailer. For example, if there are lots of details in the image, like in an aerial shot of a horde of animals or a forest scene with many leaves and branches; that would be one category or model variant. Another variant could be when there's a close up shot of an animal. We could also go back to the image used for styling, perhaps another kind of image would work better for some scenes?

Since I don't have much experience with style transfer, I cannot say what approach would be best or has the highest chance of producing better results. Perhaps it would be a waste of time to try any of the suggestions I gave, because we actually should look at another neural network architecture or maybe we just can't get noticeably better results.

What (hopefully) does become obvious here is that machine learning involves lots of trial and error. As mentioned above, it took three hours to train the style model on a 980Ti GPU. Its launch price in 2015 was $649, but can still be found on ebay for £100-200. This graphics card needs 250w, excluding the energy needs of the rest of the system. So there's a decent amount of energy use here, especially adding that we had to train for 3x3h to find a good parameter setting.

There are analyses of compute cost (open ai) and/or energy and cash cost (yuzeh) of machine learning, but they only take the final training cost into consideration. What I mean is, if they would talk about my simple experiment here they would use the 3h number to calculate compute, energy and money cost^[4]. Even if they take all self-play matches of AlphaZero into consideration, they don't consider all the testing it took to find the parameter values they would use for all those self-play matches!

But wait, there's more! We have only mentioned the trial and error procedure to find the right parameters when we have the right architecture.

What about coming up with the architecture in the first place? It's not easy, and is a combination of experience and exploration. I honestly don't know how much you can predict mathematically about the performance and output effects of different architectures, as that's not my research area. I do know that it's a very active field of research though, because we simply don't know enough about how neural networks function, leading to for example the "lottery ticket hypothesis" about using bigger neural networks. In short, it means there's trial and error in finding a good architecture as well.

It's uncommon these efforts are mentioned in papers presenting results, or even in discussions such as those referenced above about compute cost, which is unfortunate. It can give the impression that machine learning, especially using neural networks, is easier than it actually is. The resource costs to get into this kind of research are enormous, and even if you had lots of money, you would need expertise and experience to not waste those resources.

If we look at deep reinforcement learning research (deep RL) for example, this becomes even more apparent. In deep RL research, the most common task is to get machine learning systems to play video games. In a rare example of transparency (though still glossed over), the paper presenting pysc2 - a python framework to play Starcraft 2 - has a few graphs like this:

Graph from pys2c paper showing score on the vertical axis and time steps on horisontal axis. The different colours represent different architectures for the neural net.

It doesn't really matter what the task is that this graph represents, it could be any task. Each colour is a different neural network architecture, and the bold lines are the best results for each architecture. The faint lines are what I want to point out. These faint lines grey lines are using the same architecture as the winning thick grey line, but uses different hyperparameters. Meaning, parameters such as the style weight and content weight we had to trial and error for the style transfers.

So, not only did they run 100 experiments for each architecture. If we read this graph right, it looks like only two of those 100 experiments for the winning grey architecture rose high above the others. And you would not even know that until after around 200 million time steps. Additionally, if we look at averages across all the lines of each colour, doesn't it look like the orange architecture is better overall? If so, the researchers have actually chosen outliers as their presented result!^[5]

Are you starting to grasp the enormous amount of work put into something that works?

It's no wonder Deepmind and OpenAI have hundreds of top-of-the-line PhD graduates for research and millions of dollars in backing to fund the computational resources they require. Looking at recent advances in text generation, specifically GPT-3, it's an enormous computational achievement that's only feasible for large and well funded organisations.

Conclusions and up next

To conclude, machine learning and deep learning in particular are expensive activities. One could question if it's worth the climate costs. And what are the societal costs and effects when only huge companies can train these systems?

In the next post we shall touch on those questions. We will also discuss the input/output relationship we have seen both here and in part 1; how we need to test and evaluate output depending on the input. If we are not careful, we can cause real issues in society so it's important to discuss the implications of magic input-output boxes.

Edit: Now available at a server near you: Part 3

Of course, this is a highly simplified description of what happens, see Wikipedia's object detection for an overview and more links. ↩︎
There are services like Google Colab, but unless you pay there are limits on the size of your models and how long you can use it. Also you should consider if you're comfortable letting Google access your dataset. ↩︎
Thanks to @rob_homewood for sending me ready to use command line snippets, it likely saved me at least an hour of reading to find out the correct combinations :) ↩︎
And I only knew the range of parameter values that were worth testing because the readme on github specifically mentioned the range of values they used! ↩︎
This could be seen as a biased presentation of results, but I don't have their data available so I cannot say for sure and it is difficult to distinguish the colours in the graph. The paper is not peer-reviewed so this is one thing reviewers hopefully would catch. ↩︎

Planet Ghibli Part 1 - Setup and Play

foh@gamescapad.es (gamescapades) — Sat, 04 Jul 2020 02:00:00 +0200

Ever since I first saw the examples of neural style transfer back in 2015 or 2016 I wanted to see what Planet Earth would look like in the style of Studio Ghibli. Since I'm a ~~lazy~~ busy person, I've eagerly awaited someone doing this for me. Alas, here we are several years later and no such thing has come to pass. Humankind, you have failed me.

Good news everyone! I decided to take a crack at it myself. And at this point style transfer has become such a standard example that it's easy to find code for exactly what we want. However, while doing this I also realised this is a great visual example of how machine learning works in practice; why it's so heavily dependent on what data you use and what parameters you set, how it's about trial and error and waiting for training to complete, and therefore why it's often about ~~black magic~~ experience and how much resources you have, can buy, and is willing to sacrifice the environment to use.

I will not be going into much detail about the algorithm or the code, as I based this on the excellent tutorial for Pytorch where they explain the algorithm as they go along. Their conceptual overview is nicely complemented by the wikipedia page on neural style transfer.

However, I have provided a "cleaned up" version of their code on GitHub where it's easy for you to replace the filenames with your own images. This can be useful if you just want to test things without having to understand much of the code or going through the tutorial I linked to.

What I will be discussing and showing more than code is the process; how what images you select impact what comes out the other end. The trial and error process of machine learning.

Very brief explanation of style transfer that is likely to annoy people fluent in mathematical details

Let's say we have three images, the style (say, picasso) the input (for example a photo you have) and the output (the resulting combination of your style and input images). Style transfer as presented here involves using pre trained object recognition networks (VGG16 and VGG19) to calculate 'values' for the content and style for the output, style and input images respectively. When I say "content" and "style" here without writing them like this I mean the "conceptual" aspects of our images. Maybe your photo is you in a silly hat. That's the "conceptual content" so to speak. The "style" of the photo is, well, a photo, so it looks like "reality" (whatever that is). Whereas our style image is a painting so regardless of what the painting supposedly depicts, it's a painting and not a photo. Get it? Great!^[1]

So the algorithm can exploit the fact that object recognition neural networks have many layers and depending on what layer we decide to look at we can extract and manipulate the "content" and "style" separately.

So, say we start out having the output and input images being the same; your photo. We calculate a "distance" between the content of output and input which of course here start out being 0. Let's call that C. We then calculate the distance between the style of output and style, let's call this S. Here's the magic: the algorithm now tries to minimize S while keeping C as low as possible. So the pixels of your photo are slowly being manipulated so that it's still you in a silly hat but it will look more and more like a picasso painting.

Let's get this stylin' goin'

Assuming you have downloaded my code, the main part starts at or around line 230. We load a style image, and define a directory where we have a bunch of input images.

if __name__ == '__main__':

    style_img = image_loader("style/yourstyle.jpg")

    input_dir = Path('./input_images')

Put just one image in the folder the first time, so you know how long one image takes to style. On my crappy old dual core i5 laptop it's around 2 minutes for 128 pixel resolution. Some of the larger resolution images in this post have 456 pixel resolution which took around 35 minutes on the same laptop. With a graphics card that supports CUDA it'll be much faster. To change the resolution used, check around line 22 in the code and set the resolution depending on if you have CUDA available or not:

imsize = 720 if torch.cuda.is_available() else 128

So, say we have the following image we want to style (i.e. put this image in the input directory defined in the code):

But what style image to use? Here we bump into the first issue. It turns out you can't just use whatever image you want as the style. I mean you can, but it won't look any good. It has to be carefully selected.

I duckduckgo'd for images to use and most are stills of characters. Which makes sense, you know, because there are characters in these films. Anyway. Princess Mononoke is cool, let's use a shot of her for the style. Here's what happens when used to style the above shown :

"Style" image to the left, "output" image to the right. Click for larger versions.

Okay, so it's not that this isn't kinda cool. The bottom right looks good, the main features of the content are kept and has been stylised. But the upper left has lost most of its features, i.e. its elephants can't really be seen anymore as they've all been taken up by the whiteness of the wolf from the style image. What seems to happen is some of the content of the image that we use as style is slipping through.

What happens if we instead find a style image that has fewer features of its own, like a landscape image? This turns out to work much better:

"Style" image to the left, "output" image to the right. Click for larger versions.

Now we get a result where the content is better preserved, we can distinguish each elephant in the upper left of the picture which we couldn't in the previous example. The style image from Nausicaa Valley of the Wind happens to have similar colors, which also helps. But importantly: it's the exact same code! There's a reason the tutorials for style transfer use paintings that are fairly uniform and distinct in style: they produce much nicer results. But the tutorials rarely mention this, from what I've seen, so I bet there are people out there wondering why it doesn't look as nice when they try using their own images. There's nothing wrong with your code, it's just that the algorithm isn't as magical as it may seem! :)

Now that we've found a style image that works we can try on a few more examples to see if it works for them as well.

Original image to the left, styled image to the right. Click for larger versions.

Looks pretty cool! Let's do another one:

Original image to the left, styled image to the right. Click for larger versions.

Sir David has never looked better!

To summarize, we see it's important what data you have and how you use it, the algorithm is only part of the story. I should also mention that sometimes the result is pretty different in 128 pixel resolution compared to higher resolutions. So you have to spend time waiting for styling to happen in order to explore results. Machine learning is thus costly in time, money and energy use, since you have to wait around even if you have bought fancy graphics cards and fast processors. We will see more examples of this in the next part, where we style an entire video; a Planet Earth trailer.

Until then, go forth and style!

Edit: Now available at a server near you: Part 2 and Part 3

Please ignore paintings in realistic style to make this example work ↩︎

Access apps like Jupyter notebook remotely using SSH tunnels

foh@gamescapad.es (gamescapades) — Thu, 16 Apr 2020 02:00:00 +0200

Hello there. I hope you and your loved ones are safe in these disturbing times.

I've been working from home since early March now. Currently this work mostly involves writing Python code for simulations and modelling of data. My laptop is fine for the initial experimentation, but at some point I need more computational power to do 'big runs' of my models. That's when I connect to a machine in my office by typing http://localhost:9999 and I get a full jupyter lab session that's running on the office machine.

What? I can access a remote machine by writing local host? Yes. This is the magic of SSH tunnels.

Maybe you already use TeamViewer or similar software and don't see the point of this exercise. Well if that works for you, great. But the advantage of this method is that much less information has to travel over the interwebs. With TeamViewer you send the entire picture of the desktop so it can often be unresponsive, especially if you're on a bad connection. With SSH tunnels, results can still be slow to arrive, but at least the image won't become huge pixels where you can't see the text anymore :)

In case you don't care for long explanations there's a TL;DR towards the end.

Some prerequisites

If you've never heard of SSH this tutorial might be tricky. But if you've ever used GitHub to store your code you've probably already used SSH. However I will assume basic knowledge of SSH here, so if you lack that basic knowledge, you can for example check out this tutorial at Digitalocean and then come back :)

Another concept you need to be familiar with is that of ports. Most computers and networks today have firewalls so you cannot access them without opening ports, or doors, into their juicy innards. Web sites are accessed over ports 80 and 443 so if you run a web server those ports need to be open. To access a computer by SSH, the default is port 22. Digitalocean generally has great tutorials, and here's one about firewalls and opening ports for SSH.

Oh, and our main prerequisite is that you are able to access this remote machine over the interwebs via SSH. We will also look at the case where we cannot access the target machine directly, but have to 'jump' through another machine first. That's what I have to do; I first SSH into a publicly available machine on the university network, and from there, I can then access my office machine with another SSH connection.

I will also switch between using the terms machine and computer, FYI.

Running an app locally

Before launching a site on the interwebs, we usually develop it locally first. Blog applications like Jekyll have a built in web server that automatically reloads any changes you make. So when I'm writing this blog post, I have a browser window open pointed to http://localhost:4000. Every time I save, Jekyll automatically rebuilds the site and I can check out what my post looks like now and then while writing.

The localhost part means your local computer. This is usually interchangeable with the IP addresses 127.0.0.1 and 0.0.0.0. I say usually because there are some exceptions but those are not really important for our purposes today. Just be aware of it if you see those IPs when you're poking around getting these things to work.

And the 4000 is the port Jekyll has attached itself to. So localhost:4000 means you tell your browser to "go connect to my local computer, find door 4000 and there shall be magic sights there".

If you have Python installed, you can test this very easily by opening your terminal and change to any directory you want, like the one where you store all your cat pictures. Then you can start a web server from there:

cd whatever/folder/where/you/have/cats
python -m http.server

It will most likely tell you it's now serving HTTP on port 8000. Go to http://localhost:8000 and you should see a list of all those cat pictures. This nifty little python command is very convenient for testing what we will be doing next, in case you don't have or need JupyterLab. You can select what port will be used by specifying it at the end of the command, like so:

python -m http.server 12345

Many applications these days use a browser interface, and a very common one in the Python world is JupyterLab. It's not my personal favourite when it comes to code editors, but it's very convenient to have it running on the beefy machine in the office and I can access it from the comfort of my office desk by just pointing my web browser to the beefy machine (we call it the Beast, because it's got beastly powers. Well, it did a few years ago, it's actually not that powerful compared to what's available today, but let's not hurt its feelings. It will always be the Beast to us.).

But what if I'm not on the same network, or there are firewall restrictions, or I'm far away in the comfort of my home?

Accessing an app remotely

The magic of SSH is that we can connect through its port, and bind local ports on the remote machine to local ports on the computer you're sitting at.

This is going to be more tricky to keep track of pretty soon, so let's recap what was just stated. Every computer has its own local ports. They are local to that computer. What SSH can do is that it can connect a local port on some other computer to a local port on your computer. No need to open any more ports than SSH!

Okay! Open a terminal window, ssh into the remote machine and start JupyterLab. We need it running or there's nothing to connect to, and especially for testing it's nice to have it running so we can see if someone (like ourselves) connect.

JupyterLab by default runs on port 8888. So what we are going to do is bind the port 9999 on my laptop to 8888 on the Beast. This is also called port forwarding. Like forwarding mail if you've moved temporarily, you put a temporary address to your temporary house^[1].

Diagram showing localhost:9999 on laptop connected to localhost:8888 on the beast over SSH. Created with mermaid

Now open a new terminal window. Assuming you connect to your remote machine over the default SSH port, all the magic happens in a single line in your terminal:

ssh -f -N -L localhost:9999:localhost:8888 user@remotemachine

Let's go through those commands we give ssh one by one.

-f means we tell ssh to go to the background. We don't really want to do anything on the remote machine other than set up the port, so this way we can issue more commands in this terminal window or just close it to keep our workspace clean.

The -N command means we don't execute any commands on the "other side" of our connection. Again, we only need to set up the ports so we don't need to execute commands on the remote machine.

And then we have -L which is the command to bind our ports and we give it the instructions to bind localhost:9999:localhost:8888.

If you now go to localhost:9999 in your computer's browser you should see JupyterLab running on the remote machine! (Or whatever application it is you might be running) If not, is the application actually running on the remote machine? Forgetting that step is the usual error for me :) Otherwise, try going through the steps again from the start.

Crystal clear? Super green? Good. Things will get more complicated now.

Multiple tunnels

Now, let's say our Beast machine cannot be accessed over the interwebs. But it can be accessed on the internal network of our company or university. And there is some other machine at our place of work, a machine that is both open to the public web and the internal network.

So to do anything on our Beast we have to first - let's call this computer Unicorn - login to Unicorn. From there we SSH into the Beast and start our JupyterLab session.

Now, we can only bind/forward ports on our local machine, so we need to do this in two steps. First we create a tunnel to the Beast, and then we can do the same connection we did in the previous section. Let's look at it step by step.

Multiple tunnels: step 1

Diagram showing localhost:12345 on laptop forwarding to beast:22 on Unicorn over SSH. Created with mermaid

The first step is we bind a local port on our home machine that works as a shorthand to connect to the Beast as if we are on Unicorn. What we then have is a local port that automagically connects directly to the Beast through Unicorn.

ssh -f -N -L localhost:12345:beast:22 user@unicorn

So what we do here is not actually connecting to the Beast, we only say that localhost:12345 - meaning port 12345 on our laptop - can be used to connect to the Beast via ssh. If you run the above line, exchanging the values for your own addresses and users, you'll see that you only get asked for the password for user@unicorn. (If you're using SSH keys this is not as clear, it'll just work if everything went well.)

To test that this works, you should now be able to connect to the Beast through the local port we created:

ssh -p 12345 beastuser@localhost

Here's where things might become confusing, because as you see you have to use your Beast user account @localhost. Which is because our local port 12345 is now actually our forwarding address for the Beast! If we again use the analogy of forwarding mail to a new house, the tunnel we have created is just the forwarding address. Connecting through that address is like actually sending mail to the new house.

Got it? Great! (If you don't, play around with different port numbers until it clicks)

Multiple tunnels: step 2

Cool. Almost there now. This final step is very similar to what we did in the "single hop" scenario where we can connect directly to the Beast through SSH. Thanks to the tunnel we setup in step 1, we now have the same situation, only with a slight twist:

ssh -f -N -L localhost:9999:localhost:8888 -p 12345 beastuser@localhost

Everything is using localhost now! You said a "slight" twist! What is going on?!

Calm down. You get this if we just look at it in parts.

So the last part, -p 12345 beastuser@localhost is the same as when we did the test connection in the previous step, right? We connect to the Beast using our new forwarding address localhost:12345.

And by doing that, we can access the local ports on the Beast, in this case 8888 where JupyterLab is running. And just like before, we bind our home computer/laptop port 9999 to the Beast's 8888.

In other words, now we can use JupyterLab from the comfort of our own browser!

Diagram showing localhost:9999 on laptop forwarding to beast:8888 through Unicorn server. The direct arrow is our end result and the upper arrow is the path we actually take. Created with mermaid

Summary and more possibilities

So now you know how to tunnel like a mole. Pretty nifty, huh?

These basics allow for more uses. Just three examples:

Mount the filesystem of the remote server and browse it as if you're using a local disk using SSHFS (SSH file system)
Use graphical applications of a Linux app using X forwarding
Use a cheap cloud server in another country and tunnel through to watch local video streams

There's also the concept of reverse SSH connections, in cases the machine you need remote access to is behind a firewall and there are no other servers on the network you can SSH into. I might write a guide for that in the future, depending on how eager I am to increase my post/design ratio.

Hope you find all this useful. I do almost every day!

TLDR

App running on port 8888 on a remote machine can be accessed on your local machine's port 9999 with:

ssh -f -N -L localhost:9999:localhost:8888 user@remotemachine

In case you need to multihop through a "jump server" we need the extra step:

ssh -f -N -L localhost:12345:remotemachine:22 user@jumpserver

Followed by:

ssh -f -N -L localhost:9999:localhost:8888 -p 12345 remotemachineuser@localhost

Troubleshooting

If your internet connection drops, the ports and bindings are sometimes not cleaned up properly. You'll then be unable to access your application through the browser, but when you try to establish the SSH tunnel you get errors saying the port is in use.

On Mac and Linux you can issue the following command to check currently running ssh processes:

ps aux | grep ssh

If you have no active ssh sessions this will most likely output only two lines, one saying something about ssh-agent which is a background process keeping track of your ssh-keys if you need them. The other would be saying something about grep because it shows the command we just issued.

But if you do have active connections these should be listed here, and their process ID should be the second column from the left. Meaning you can kill those processes manually with:

kill 12345

If you have 'multi-hopped' then you can save yourself a few keyboard taps by killing the process of the first port binding in the chain, the others should drop automatically.

Yes, nobody sends mail anymore, but I think analogies tied to the real world are easier to handle than the abstractness of digital things. ↩︎

Creating Websites Like It's 1999

foh@gamescapad.es (gamescapades) — Tue, 25 Feb 2020 01:00:00 +0100

Around Christmas I launched a new personal website. It's the first time in years I've written all the code for a website from scratch. All those new CSS things with responsive design and media queries scared me off so for this blog and other sites I've mostly depended on themes made by others and then customized them a bit. Then I came across a great summary/overview of CSS and was inspired! So now I've also changed this blag's design a bit, the main addition being a dynamic header menu that stays at the top when you scroll^[1]. I can find these annoying as they reduce the amount of space for text, especially on the phone, but I think it works fairly well here as it's now easier no navigate around even if you've scrolled to the middle of a long blog post.

I originally also planned to convert this blag from Jekyll to Gatsby as the latter is apparently very fast and search engine optimized and all that jazz. But when I started reading the documentation my eyes glazed over. Why all these contortions just to get a simple blog up and running? I'm sure Gatsby is great for someone out there, but in my mind it's only great for those already brainwashed in the horrible mess that is Javascript package management.

Image stolen from random reddit thread.

However I did find 11ty which looked nice, and more recently updated than Jekyll. They have great little animations making fun of buzzwords and a cute possum on their website. That's enough to sell me. I love that you can run it from basically nothing without any setup other than installing and creating a new directory. What finally put me off using it was mainly that their plugin ecosystem isn't as strong as Jekyll's yet, in particular the syntax highlighter could need some work. But when/if I decide I want to give myself ~~more work~~ a fun hobby, I'll definitely look into it again. Jokes aside this current setup with Jekyll would really need some dynamic resizing of images so download size can be adapted automagically when you're on a bad connection.

When I was seventeeeeeeeen

I used to create and run websites in the late '90s and early '00s. If I remember correctly I learned HTML by copy-pasting from other websites and magazines. It's difficult to do that these days as the source code for most websites is obfuscated by ad tracker javascript. Back then CSS was really new or even non-existent so we did silly things like designing with <table> and using 1x1px images to align elements. When CSS came around we could style the table borders to align things instead. Yes it was the stone age.

I'm sure you're dying to know what kind of websites I used to create.

Mostly it was video game fansites. One of the more interesting ones was 'the Games Comparison Archive' where I wrote articles comparing games in the same genre, as well as collected and normalised review scores from game review sites. Unfortunately Wayback Machine doesn't have a snapshot (likely because the site didn't last very long) and weirdly redirects to some dos games archive but I'm 99% sure this was the URL: gamesarchive.etmnet.com.

Too bad I didn't continue, I heard metacritic was fairly successful.

The coolest thing I did was probably a Swedish fansite for Warcraft 3. I started it in May 1998, one and a half years before the game had even been announced. The first version was just a wishlist for what I wanted Warcraft 3 to be. Then I started getting emails from random people, who wanted to share their own wishlists. So I put theirs up as well. I started posting news, and as the rumour mill of the actual game wasn't running very quickly, the news were mostly about the wishlists and what happened on other Warcraft fansites. It was fun and kind of weird to know there were other nerds like me out there, and be able to connect. I managed to find web hosting from ETMNet so I could use Perl CGI scripts to update the news more dynamically than editing and uploading HTML all the time.

At some point I figured all these nerds sending me wishlists might want to talk to each other and not just me. So I found a book about PHP with a tutorial of how to program a forum. My foundation skills in programming started there, and to this day I still believe the PHP documentation is one of the best ones out there. If nothing else for the fact they've got a comment system where people shared tips and tricks. Kind of like Stack Overflow but much earlier. The site grew, the forums got more active and I even hired people to help out. I paid them in nerd sweat. (If you, dear reader, happen to be one of them then please get in touch! It's been too long :)

I redesigned the site a billion times. The oldest version I can find on Wayback machine is one from April 1999, five months before Warcraft 3 was announced in September 1999.

And here's what it looked like in 2001, when I apparently had gotten my own domain name.

I closed and reopened the site many times as my motivation went up and down. The funniest 'closing message' I remember was one that went something like:

"If motivation could be bought on a can, this wouldn't be a problem. Unfortunately, such is not the case."

Somewhere around when World of Warcraft was announced, me and the fine people of diabloii.nu, a Swedish fansite for the Diablo games, got together and created warcraft.nu. It was likely Sweden's biggest Warcraft fansite at some point, we had at least 10k visitors a month if I recall correctly.

I lost interest around 2005-2006 or so, I can't remember exactly when. But the others kept the site going for years after.

Looking back at it now, I cringe at some of the things teenager me wrote. Not to mention my nickname, I'm dying of embarrassment. But mostly it's fine. Sometimes even funny. And also it was 20 years ago. I suddenly feel old. The interwebs were cooler before they became cable TV.

ps

If you're wondering about the .nu domain it was commonly used among Swedes because 'nu' means 'now' in Swedish (and in a few other languages). Also I think it was difficult to get a .se domain name as an individual back then so .nu was the common way to go. Apparently the country Niue sued Sweden's internet foundation in 2018 for taking control of the domain name.

Awkward!

This is no longer true as of April 2020, I changed the design again! ↩︎

Mirror's Edge Catalyst is a Crap Game

foh@gamescapad.es (gamescapades) — Sun, 19 May 2019 02:00:00 +0200

I used to have a huge 3x2 meter version of the below art on the wall of my hallway. I fucking loved the first Mirror's Edge. The art style was cool, I discovered the magic music of Solar Fields and though there were bugs and quirks in the gameplay, the concept of a first person platformer was fresh (2009). There wasn't much story to talk about, but Rihanna Pratchett did admirably, seeing how she was presented with a basically finished game when DICE hired her. And when Mirror's Edge flowed, the white and brightly coloured surroundings melted into streams of light feathered bliss.

So I was slightly annoyed that the sequel, Mirror's Edge Catalyst, was released late 2016, just when I had moved to London and didn't have access to a gaming rig. Fast forward to January 2019, when I bought a PS4 in order to create some form of work/life balance with this stressful PhD I'm doing. (Let's not mention that it took time to save enough money to justify the purchase. Oops!)

The first game I played on this PS4 was Horizon: Zero Dawn and I loved it. Not revolutionary in any way, but a beautiful and very competent game with tight and fun combat mechanics, a decent story and main character who was not only tough as hell but also wonderfully sarcastic if you chose the right dialogue options. Oh, and robot dinosaurs. I mean, they had me at robot dinosaurs. I only really have two real criticisms of Horizon: Zero Dawn. The first being that cliffs are only scalable at certain points, and these points can be difficult to find, so several times I spent lots of time running along cliffs trying to find that point only to sometimes conclude there was none. The second criticism is that I initially ran around randomly to explore the world, only to find big areas completely empty, later realising I had to activate a quest for these areas to be populated. But those things don't matter, because the rest of the game is amazing, just look at this:

Aloy kicks ass. Dinosaur ass. Clip courtesy of SunhiLegend

Mirror's Edge Catalyst however, is not amazing. Mirror's Edge Catalyst is a worse game than the first one in all areas. I'd go so far as saying it's a worse game than most games. I can't believe that this game is the result of any reasonable designer's actual vision. It's a mess of half-assed things that all scream "me too!". The kind of metoo that managers give orders about because all they do is read feature lists of the top games on metacritic.

"It has to be open world! Lots of side missions with map icons like how Ubisoft does it! Players want combat! We need to monetise! There has to be some kind of multiplayer interactions!"

--random marketing exec

I really hope, for the sake of all the people who worked on this game, that this crap shoot of a game is the result of management being idiots. It’s like a game from I don’t know when but absolutely not 2016. If you want to copy other games, why did you not look at anything that happened between the first Mirror's Edge and 2016? Texture resolution is so low the game looks like it's from before even 2009, and there's still texture pop-in. I would've been satisfied with a free runner with tight controls but not even that part of the game is polished. Running and jumping was generally fine though, until I got to an area supposed to be a big mainframe of some kind. I jumped into invisible walls and needed several tries and very particular aiming to progress. Apparently the test department got tired somewhere around here. Or fired.

I mean if it’s open world; why not allow us to see some of this world and its actual inhabitants by having some levels set on the ground or at least among civilians? Parkouring through a shopping center filled with people, up and down escalators and statues, or jumping from car roof to car roof or.. so many missed opportunities and instead this mess where the only random people you bump into are these people standing around extremely weird places giving side missions. They can be placed in the middle of fights, next to explosions, in ventilation shafts and just generally nonsensical places.

The only way to get some information about what's happening in the game world is through TV screens. But more often than not, the sound of these TV screens is drowned out by characters also standing next to the TV screen, and they're saying random things because you’re standing close. If you step away from the character to avoid this happening, you can't hear the TV anymore. Good job there.

Speaking about thinking things through, as usual there are helpful tips displayed on loading screens, exemplified below:

You don't always have to stay and fight. This is technically correct. Because often, you cannot progress without fighting. But not always.

This text makes sense, it’s a game about free runners after all. Well, unless it’s a boss fight, where you have to fight waves of enemies. Which is stupid for two main reasons. First, it’s a game about free runners, why is there forced combat? Two, did you completely miss the whole shitshow that went down with Deus Ex Human Revolution? Where players had to fight and kill really tough bosses even if they had spent all their points into sneaking? I know, I know, that was 2011, but I had spent all my points on running when I first encountered this kind of forced battle. So why? Honestly I don't think it would matter much, because the combat system is really bad. To properly dodge you have to lock on to the target which allows you to spin around them to dodge. But the lock is so wonky it sometimes doesn't work and/or releases randomly. And if there are multiple enemies you can't dodge them, because only the enemy you're locking on to can be spun around. Why not just have a dodge roll?

It would make sense because, oh I don't know, rolling is what Faith does every single time after landing from a high fall? So yeah, these boss fights are completely ridiculous. In the case of Deus Ex Human Revolution, Eidos' excuse was that another company made the boss fights. Not that that absolves them since they still chose to put those boss fights into the game, but what’s your excuse, DICE? Oh right, you're still stuck in 2009. And everybody knows the dodge roll was invented in 2014 by the developer Dodge Roll, how else would they have gotten that domain name?

Speaking of loading screens. They go on forever, probably because of bad optimisation (remember texture pop-in mentioned earlier?) I also saw them way too often. Because I kept falling to my death. Which is fine, if it's because I screw up. But most of the time, it was because of bad hit boxes, imprecise controls and extremely unclear affordances (hints; visual or auditory markers). Like here:

The red is a lie. We want to get to the platform over to the right in the picture.

The pipe is red, thanks to "runner vision" which paints red those objects that can be used for climbing and jumping. However, if I climb up it and try to jump onto the platform to the right in the picture, I won’t reach it and will fall to my death. Meaning loading screen.

The red is not a lie here, surprisingly enough. This is a view from the far side of the room seen in the previous picture. The platform in front of us is the same middle platform we wanted to get to in the previous picture. We can make out the grey pipe from the previous picture waaay over there.

The correct action is to jump from the fence/railing. If I use that to go to that other side then the railing/fence is correctly marked as red. There is no pipe here, but apparently the existence of one in the middle platform screws up the runner vision. This kind of thing happens way too often in Mirror's Edge Catalyst.

Here's another fun example:

The red is not exactly a lie here, but very misleading. The red beam to the right can be seen from far away and is basically a vertical red stick shoved into the ground where the objective is located. The compass marker in the lower part of the screen also shows where the objective is. They're both technically correct, but not helpfully correct.

The red objective marker in the lower part of the screen tells us to go down, and the objective "beam" to the right shows the exact location of the objective like a red vertical stick through the world. But neither of these "helpful" markers show that actually I need to go left into that tent to get underground in order to reach the objective.

Affordances are also often non-existent. At one point, the platforms you’re running on start breaking. In many games, this is scripted so the platforms break slightly after you pass them, so they’re there only for dramatic effect. In other games, there are affordances that indicate when a platform is about to break. In Mirror's Edge Catalyst however, it’s completely random. I died four or five times because I fell down. I tried just running, died. Tried jumping along. Died. Randomly pressed jump here and there a few times and eventually made it. I’ve no clue what I did, which is why I draw the conclusion this section was just random. Great feedback loop to improve your play, huh?

Seriously, did anybody test this game, at all? Oh right, they fired the test department. We already concluded that. Sorry, I forgot.

Speaking of crap game design. It's possible to "hack" TV screens, for no functional reason at all. So when the villains started using turrets against me, I thought; "sweet! I can hack them and use them to my advantage!". Nope. I can shut them down but not hack them. Maybe I was stupid and hadn't put points into hacking. Nope. There's no hacking skill. There was an opportunity here to allow for non direct violence but nope. Oh well, it's not like the entire story is based around you helping a hacker to take down an evil computer system. Or that the game opens up with you meeting some hacker dude who sees you as his big sister. So you know, hacking is not a skill that is easily accessible to you as a protagonist.

You think that's not really game design, it's just story or setting or whatever? How about all the times you're running through literal storms of bullets and enemies and reach an elevator/door button, press it, and suddenly all enemies are gone? Apparently they can be teleported but when I fall down the side of a building I have to see a loading screen. Poor enemies, I imagine they have to see the text "you won't be executed if you kill the player before they reach the button" on the loading screen for NPC heaven.

And these random documents and recordings scattered all over the place. What is this, the '90s? System Shock started this a long time ago (afaik), and it was kind of working although sometimes awkward as story/setting explainers. But at least it made kind of sense that these recordings were made, within the setting. This is the case in Horizon; the recordings you find are lab notes or diaries that you find where appropriate. For example, you can find lab note recordings...in a lab! There are recordings in random locations of the world, but from what I can recall, they are usually found in the rubble of old buildings. In Mirror’s Edge Catalyst however, you find a recording of a very sensitive conversation between two main characters...in a random ventilation shaft.

Sure, that could've made sense if that was the only recording found in this way. Because maybe someone secretly recorded the conversation and then hid it in a ventilation shaft. But this particular ventilation shaft is nowhere close to anything, it's not mentioned by anybody that this conversation has been recorded and was hidden, and it's just as randomly placed as all other recordings and secret documents you find. It's like someone was running out of time and placed all these collectibles everywhere and nowhere without thought.

Without thought, yeah, that pretty much sums up how Mirror's Edge Catalyst must've been created.

EA/DICE turned into a Battlefield making workshop many years ago now. I haven't really played any of them much since Battlefield 1942. Wait that's not true, I played Battlefield 3 for several hours. Oh, but 1942, that was a great game. What a multiplayer demo. I guess EA/DICE are incapable of doing anything innovative these days. Or even slightly different than run-of-the-mill manshooters. Which isn't that surprising, they haven't tried leaving their comfort zone in years. And seeing how crap Mirror's Edge Catalyst is, they never will again. You keep making your Battlefield FPS games, DICE. It's okay to be a one-trick pony. At least your shareholders are happy.

Compare all this to Horizon: Zero Dawn, which as I mentioned is not a revolutionary or even innovative game. But it’s great because the parts they have are well made, make sense in their context of the world and are actually interesting. The developers Guerilla Games must've had lots of experience making this kind of game before. Oh, they don't? They're mainly known for their FPS series Killzone?

Well, isn't that a funny coincidence?

Peace out bitches.

Building Bots in StarCraft 2 for Psychologists

foh@gamescapad.es (gamescapades) — Mon, 04 Sep 2017 02:00:00 +0200

Hello visitor! I haven't blogged in many years, so I thought I'd kick start a new era of blogging with a tutorial series on how to make bots with the recently released pysc2.

I'm not a great programmer in any way, but neither is this a tutorial on how to program, so I will assume you have basic knowledge of python. You will likely also want some familiarity with working with matrices in python via numpy, but maybe not. I actually explain many of the numpy things in some detail, since the Deepmind coders do stuff in one line i would need a bunch of lines to do. But please tell me if there's something you want me to explain further!

Without further ado, let us get cracking.

Introduction: humans are amazing

Imagine yourself standing on the street. You take in your surroundings, your senses send signals to your brain, while your brain sends signals back to your senses, telling them what they should see, and you decide to...

Well, what would you like to do? So many things you could choose. That thing about brain sending signals back to your senses is a real thing by the way. And there are more of these connections than those going from your eyes to your brain, implying that we indeed see what we want to! (See here for an example).

Makes you think doesn't it? (pun intended) However, we shall not dwell on that subject more (although it is part of my research if you'd like to know more).

So, back on the street again, let's simplify the scenario. You can only take discrete steps. For every step you take (and every move you make), you look around and decide on an action based on what you see (we shall disregard imagination for now, you only make decisions based on your surroundings). What would you do and how would you reach that decision?

This is still a very complicated scenario, as the real world is filled with many wondrous and horrible things and we can choose to interact with our world in even more combinations.

Let's simplify even more, to the scenario of playing a video game. This is a much more constrained space of possible observations, it's governed by rules we know and the actions we can take in that game are pre-defined.

In my field of research, we use video games as experimental settings to try and understand how humans make decisions and use that knowledge to improve the capabilities of artificial intelligence.

Which takes us to StarCraft 2. It's a so called Real Time Strategy (RTS) game, where you gather resources to build a base and enough units to destroy your enemy. It's like Chess but in real time and way more complicated as the combinations of possible things to do are way more than in Chess.

It's a lovely game, I especially like the colors and sounds of the Colossus unit's energy beams as they disintegrate pathetic Zerglings trying to come close enough to bite.

This is not a Colossus disintegrating Zerglings, but I think giraffes are more amusing anyway.

How do we learn to play a game like StarCraft 2 (SC2)? If you think about it, it's crazy that anyone can learn to play something that complicated. Yet here we are, with huge tournament prizes for being the best SC2 player in the world. And, you know, since we can learn to read and write, play football, play musical instruments, climb Mount Everest and go to the fucking moon, maybe playing SC2 isn't that big of a deal, right?

Actually, all of our assumptions so far have relied on that we know how to interpret our sensory signals and how to perform actions like moving our arms, pressing buttons and so on. Which is something babies have to learn how to do (and there are different opinions on how much innate knowledge we have at birth).

That's not even the whole story. If you take the approach of embodied cognition, human behavior is not just input-brain-output. It's a brain in a body in an environment and our behavior is an emergent phenomenon in the interaction between those three systems (which themselves contain many different subsystems).

So, you know, to create an experiment closer to how humans learn to play StarCraft, we should do something more complicated. Like, creating a Minecraft bot that learns how to use its body, navigate and survive in its environment, and then learns to play SC2. An artificial intelligence agent embedded in its world (Minecraft), learning to play a game (SC2).

Confused? Yeah, now you know why there's so much confusion in the psychology and neuroscience fields. It's a complicated subject. Now that you know that, building a bot will feel so easy :)

So forget all I just said. Don't think about the incredible complexity of human behavior for a moment and lets get down to the business of creating a bot that can play SC2.

pysc2 - a love story between Blizzard and Deepmind

If you're not familiar with StarCraft II, I highly recommend trying it out. Not only is it fun, but you'll also be able to understand this tutorial better. You can try the free Starter Edition on Battle.net. You can actually use the free starter edition for this tutorial!

pysc2 is Deepminds python wrapper of Blizzard's sc2 api. This is nice for people like me who don't know much about C++, and also because python has access to so many great libraries for neural networks and data science. Additionally, pysc2 was made specifically as a reinforcement learning environment. To know more, read the paper. I don't think you have to read that to follow along in this tutorial but it will likely help you get a feel for how the framework...works.

However, an issue is that the documentation for pysc2 is not great. It mentions the main concepts but is low on detail, so for a fool like myself it was not easy to understand how to go about making my own bot. But I want to use this in my own research so I need to understand this whole thing. And might as well share my findings in case it helps anyone.

Installation

The instructions on the official pysc2 Github page are clear and helpful. It's fairly easy to get started if you ever used pip before to install something.

But in short, if you're on Windows or Mac, install StarCraft II with the standard installer. If you're on Linux, there's a special version for you on Blizzard's Github.

Then you need python, which you can download from the official site. The latest version, 3.6, should work fine.

When you have installed those two, open a command prompt (Win) or terminal window (Mac) and do a:

$ pip3 install pysc2

(Or just pip if you have python2) When that is done, download the maps and Deepmind mini games and extract those to your StarCraftII/Maps/ folder.

You should now be able to do a test run using:

$ python -m pysc2.bin.agent --map Simple64

That's it. This procedure was all I needed for installing on a Windows 7 machine. Your mileage may vary, as usual when computers are involved.

Overview of the framework

Following on from the story about yourself walking around the streets making people uncomfortable, it's worth repeating the three basic conceptual terms we're going to refer to over and over again:

agent - yourself in the street walking analogy. Here it is another name for bot, which in turn means automated player
observation - your surroundings on the street at a discrete point in time. The state of the world (game) at a certain time point
action - waving to someone, walking down the street. Commands sent to the game

Worth noting is that in reinforcement learning (next lesson), the algorithms usually speak of states (s) and actions (a) instead of observations and actions. But we'll get to that in the next lesson.

Other important terms are:

episode - one full training round, usually one episode is from the start of the game to its end and/or when a reward is received. So to use our walking down the street scenario, one episode could be you walking down the street and successfully getting on the bus to work. The next episode would be the next day when you try to do the same thing. This may sound silly, but maybe you just moved into this house and it's the first time you're going to the bus stop. You need a few days - a few episodes - to learn the route to the bus stop.
step - An episode can contain many steps. Each step is basically Observation -> Decision -> Action. Then we move onto the next step.
reward - Each action taken can lead to reward. Sometimes that means the end of the episode (getting on the bus can be the reward). It could also be that you get a reward for reaching checkpoints along the way to the bus stop, and getting on the bus is the end of the episode. The goal is to maximize your reward over time.

On a conceptual level, pysc2 provides an "environment" for an agent. pysc2 takes care of starting up the game, waking up your agent and works as the intermediary between your agent and the game. pysc2 gets observations from the game, sends these to your agent, and your agent sends back actions through pysc2 to the game itself.

Observations

Since StarCraft 2 is a complicated game with lots of information, Deepmind has scaled all of this down. Instead of getting the entire screen contents in full resolution, we have two main things; the screen and minimap.

The blue box is the screen and the yellow box the minimap.

The game screen we get each observation is not in full resolution. The default here is that the entirety of that game screen is scaled down to an 84x84 matrix. Which is very small and it would be close to impossible to discern any useful details in all that. So, instead of all 1920x1080 RGB pixels being scaled down to 84x84, we have what is called 'feature layers'. One feature layer for example shows only the units you currently have selected, by giving you an 84x84 matrix where 0 is background and 1 is a selected unit. Important to note is that since it's a scaled down image, one unit can cover more than one pixel/one position in the 84x84 matrix. Another feature layer shows the health of units on different parts of the screen.

The same goes for the minimap, it has feature layers like this as well, and the default size for that matrix is 64x64 (i think?). You can read more details in environment.md if you'd like.

This is the pysc2 UI, where you can see all the different feature layers on the right and a very basic graphical representation of the game to the left. Click the image for a larger version.

Actions

The amount of possible actions are potentially enormous. So Deepmind has discretized these as well. Based on actions taken by human players in thousands of games, they have defined around 500 actions. For example you have actions to select units, move units, pan around the map and so on. We will go into more detail about how these are used later, but if you're curious you can check out what they say about actions in the official documentation).

Putting it together

When we start a mini game or map with pysc2, it boots up StarCraft II and uses a run loop to go through all episodes and steps within those episodes. The run loop can be found in pysc2.env.run_loop and if we try to write that in pseudocode, the run loop in pysc2 is something like this:

number_of_episodes = 50
env = start starcraft2
map = load map MoveToBeacon
agent = setup some agent
while number_of_episodes not 0:
  goal_reached = False
  while not goal_reached:
    obs = environment.update()
    action = agent.step(obs)
    reward = takeAction(env)
    if reward == 1:
      goal_reached = True
  number_of_episodes -= 1

This isn't how it actually looks but I tried to make more understandable. Don't worry though if all of that doesn't make sense, the main thing to remember is that obs variable sent to the agent, as it contains all the observations for the current state/step. Inside of that variable we have a dictionary of all observations made, in obs.observations. This is not very clear in Deepminds environment.md so I've started an effort to update that in my forked repo version of the same file. You can find that here

Basically, to get for example all available actions in the current game state (some actions may not always be available, if you do not have any selected units you don't have access to the "move unit" command) we can do this:

available_actions = obs.observations["available_actions"]

So, now that you know that we have observations and actions, how do we put these together to make decisions? Let us look at the provided bots for inspiration and understanding.

Step 1: understanding the scripted bots

Our first thing to do is to understand the scripted agents that Deepmind has provided. We shall start with the BaseAgent class which is found in the pysc2/agents directory, as the example agents we shall look at later all inherit from this class. We will also use it as our base class when we construct our own agent in a later lesson.

Case study 1: class BaseAgent

What does the BaseAgent do? Let's take a look at the entire BaseAgent class:

"""A base agent to write custom scripted agents."""

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

from pysc2.lib import actions


class BaseAgent(object):
  """A base agent to write custom scripted agents."""

  def setup(self, obs_spec, action_spec):
    self.reward = 0
    self.episodes = 0
    self.steps = 0
    self.obs_spec = obs_spec
    self.action_spec = action_spec

  def reset(self):
    self.episodes += 1

  def step(self, obs):
    self.steps += 1
    self.reward += obs.reward
    return actions.FunctionCall(0, [])

The first few lines are importing stuff from the future, in case you're running an older version of python. We don't really need to care about those. We're all using python3 now right?

Anyway, the next line is slightly more interesting:

from pysc2.lib import actions

The actions module is important, as it's from here we gain access to all the possible actions we can do like selecting units, moving units, looking around the map and so on. But maybe that was obvious. All available actions in the base version of pysc2 can be found here and a general explanation here in the official documentation.

Let's move on to some actual interesting stuff, namely the BaseAgent class itself and the setup function:

class BaseAgent(object):
  """A base agent to write custom scripted agents."""

  def setup(self, obs_spec, action_spec):
    self.reward = 0
    self.episodes = 0
    self.steps = 0
    self.obs_spec = obs_spec
    self.action_spec = action_spec

The setup function is called by the framework to create an instance of the agent as we launch the game itself. It sets up internal variables for the total amount of points for reward, number of episodes and number of steps. Pretty straightforward.

The obs_spec and action_spec are called from the environment run loop and are there because different environments may have different observation and action specifications. We don't really need to care about those right now.

Next up is the reset function:

def reset(self):
  self.episodes += 1

This is called by the pysc2.env.run_loop before an episode starts, to increase the number of episodes this agent has experienced.

Finally, we have the step function:

def step(self, obs):
  self.steps += 1
  self.reward += obs.reward
  return actions.FunctionCall(0, [])

Every step that pysc2.env.run_loop (the environment) makes, the agent is sent the observations (obs) made in that step. The agent increases the number of steps, increases the reward with the one observed in the environment (0 if last step did not lead to reward and 1 if the last step did lead to reward). It then takes the action
actions.FunctionCall(0, []).

(For the observant and insightful reader you may wonder how many game frames that obs contains, every 1 frame rendered by the game or what? Well, it's complicated. In short, each step size is related to actions per minute. We may go into further detail in a later lesson, but for now you can read "game and action speed" in the official documentation.)

We'll get back to what that action call means in a minute. First let me explain that what this function basically does is the same as if you imagine yourself going out, closing the door behind you and walk out onto the sidewalk to stand there, staring into thin air. That's what we've gone through here, you walking out is the setup function. And now, you take a "step". Which means, you look around, your senses make observations, and you perform an action. Maybe that action is to take an actual step down the street. Maybe you wave at your neighbor across the street. Maybe you turn around and go back home because the world is a scary place.

How do you make the decision of what to do? Well, that's what neuroscience and psychology is trying to explain, so we won't go into that right now (even though this is also part of my research).

In our case, that part, the decision making part/logic is totally missing here. Which takes us to the action made here in BaseAgent:

return actions.FunctionCall(0, [])

This is how you make an action, and why we needed to import actions from pysc2.lib earlier. The 0 is the action ID number and the brackets are the parameters for this action. Apparently, this action needs no parameters. If you've got your terminal handy, you can write the following to see all possible actions:

$ python3 -m pysc2.bin.valid_actions

(If you use python2 you probably don't need the 3 there). In the output of that command, we get all actions, each action one line on the list starting from 0. Our function call is 0 (remember it was `return actions.FunctionCall(0, [])) so we only need to see the first few lines of that output:

$ python3 -m pysc2.bin.valid_actions | grep "" <(head -n 10)

I'm sure there's an easier way to do that but it's what I managed to put together when using duckduckgo okay? That command won't work if you're on Windows so you'll just have to scroll up, sorry!

In any case, we get the following output:

0/no_op                                              ()
1/move_camera                                        (1/minimap [64, 64])
2/select_point                                       (6/select_point_act [4]; 0/screen [84, 84])
3/select_rect                                        (7/select_add [2]; 0/screen [84, 84]; 2/screen2 [84, 84])
4/select_control_group                               (4/control_group_act [5]; 5/control_group_id [10])
5/select_unit                                        (8/select_unit_act [4]; 9/select_unit_id [500])
6/select_idle_worker                                 (10/select_worker [4])
7/select_army                                        (7/select_add [2])
8/select_warp_gates                                  (7/select_add [2])
9/select_larva                                       ()

Here, the action ID is the number to the left, then we have the action name and to the right are the parameters for that action. So we see that we have the command no_op being returned every time by the BaseAgent. no_op being short for "no operation". That also explains why it needs no parameters.

In other words, BaseAgent does nothing. That's all right for a while you know, but what if you get hungry, just standing there on the sidewalk?

We learned lots about the basic concepts of a bot, though, right?

In the next chapter, we shall look at a so called scripted bot, which cannot learn but is specifically written for one of the mini games.

Case Study 2: class MoveToBeacon

The MoveToBeacon class is a bot written specifically for solving the mini game MoveToBeacon. This mini game is very simple; you have one marine and you need to move it to a beacon, a highlighted area on the map. The marine and beacon are always within one 'game screen' so you don't need to scroll/pan around the map to find the beacon. When the marine steps onto the beacon, the agent receives a reward 1 and the game restarts with the beacon in a new place. Every step taken that does not result in the marine walking onto the beacon the agent receives a reward of 0.

Youtube video of four of the mini games, including the MoveToBeacon one. The video is not embedded, because the code to do that will inject tracking scripts and I don't want to subject my dear visitors to such nonsense. Click here to watch it on Youtube.

Here's the whole first part of the pysc2.agents.scripted_agents module, which contains some imports, some constant definitions and the MoveToBeacon class:

import numpy

from pysc2.agents import base_agent
from pysc2.lib import actions
from pysc2.lib import features

_PLAYER_RELATIVE = features.SCREEN_FEATURES.player_relative.index
_PLAYER_FRIENDLY = 1
_PLAYER_NEUTRAL = 3  # beacon/minerals
_PLAYER_HOSTILE = 4
_NO_OP = actions.FUNCTIONS.no_op.id
_MOVE_SCREEN = actions.FUNCTIONS.Move_screen.id
_ATTACK_SCREEN = actions.FUNCTIONS.Attack_screen.id
_SELECT_ARMY = actions.FUNCTIONS.select_army.id
_NOT_QUEUED = [0]
_SELECT_ALL = [0]


class MoveToBeacon(base_agent.BaseAgent):
  """An agent specifically for solving the MoveToBeacon map."""

  def step(self, obs):
    super(MoveToBeacon, self).step(obs)
    if _MOVE_SCREEN in obs.observation["available_actions"]:
      player_relative = obs.observation["screen"][_PLAYER_RELATIVE]
      neutral_y, neutral_x = (player_relative == _PLAYER_NEUTRAL).nonzero()
      if not neutral_y.any():
        return actions.FunctionCall(_NO_OP, [])
      target = [int(neutral_x.mean()), int(neutral_y.mean())]
      return actions.FunctionCall(_MOVE_SCREEN, [_NOT_QUEUED, target])
    else:
      return actions.FunctionCall(_SELECT_ARMY, [_SELECT_ALL])

Wow, this looks way more complicated, right? Calm down, it's fine! Take a deep breath, relax, and we'll work this through together.

First off, we import numpy:

import numpy

The convention is to do import numpy as np. Which is what they do in pysc2.lib.features so likely an example of different people coding different parts and they haven't really consolidated the code base properly. Which is why we're doing this tutorial/documentation because the whole thing feels slightly unfinished. Doesn't really matter though, it's just code nerdery. If you didn't know, numpy is a library for mathematical functions. Linear algebra. Matrices. You know, stuff you calculate other stuff with.

Next up, we import the base_agent class:

from pysc2.agents import base_agent

That one we know! It's the class we looked at earlier, so we're familiar with what this does.
highfive!
Moving on:

from pysc2.lib import actions
from pysc2.lib import features

We went through the action module earlier, as it's used in the BaseAgent class. We need it to be able to call and perform actions. The features module contains information about the feature layers; the observations we/the agent can make. This is not strictly needed, as we could enter things manually, but as you'll see presently, this way we make the code more flexible/dynamic.

Now, let's go through the constant definitions one by one.

_PLAYER_RELATIVE = features.SCREEN_FEATURES.player_relative.index

Here we assign a variable for the index of feature player_relative. If you recall from earlier, the screen features can be accessed from the obs.observation dictionary like so:

obs.observation["screen"][5] # where 5 is the id for player_relative

Basically, getting the index from the features module like this and assigning a variable to it is equal to us just doing:

_PLAYER_RELATIVE = 5

So what is player_relative you ask? Well, if we look in the official documentation, the screen feature player_relative means;

player_relative: Which units are friendly vs hostile. Takes values in [0, 4], denoting [background, self, ally, neutral, enemy] units respectively.

So, what we get from player_relative is a matrix of size (84, 84) representing the game screen, where 0 is background, 1 is self and so on.

Now that you know that, the following three lines should be fairly obvious:

_PLAYER_FRIENDLY = 1
_PLAYER_NEUTRAL = 3  # beacon/minerals
_PLAYER_HOSTILE = 4

Here we assign variables/constants to the relevant features we are interested in to solve the MoveToBeacon mini game. Well, actually the hostile part is not used but it is defined here as scripted_agent.py also contains classes for two other mini games. We'll get to those later. So, the two variables we are interested in right now are _PLAYER_FRIENDLY and _PLAYER_NEUTRAL.

_NO_OP = actions.FUNCTIONS.no_op.id

Remember earlier when we used the terminal to find the no_op action in the list of all actions? Here, if we know the name of the action we want to use, we can get its id (0) by using the actions module. This way, as long as this action will be called no_op in the future, we don't need to remember its specific id number. It's also easier to read the code with a variable name than numbers everywhere.

_MOVE_SCREEN = actions.FUNCTIONS.Move_screen.id

The Move_screen action is not self explanatory, or at least it wasn't for me at first. It could mean move the game screen, which is what I first thought it meant. But actually, it means moving a selected unit to somewhere on the current screen. We can figure this out by looking at the definition in our terminal window:

$ python3 -m pysc2.bin.valid_actions | grep "Move_screen"

Which gives us the output:

19/Scan_Move_screen                                   (3/queued [2]; 0/screen [84, 84])
331/Move_screen                                        (3/queued [2]; 0/screen [84, 84])

In our case, it's the 331 we're looking for. To the right we see that it takes two arguments; queued and screen. The first is a boolean and indicates if the action should be taken now or after previous actions (more on that later). The second argument is a point/pixel on the screen, which is (84, 84) pixels big. So since it's a single pixel being asked for, it must be moving a unit to somewhere on the current game screen. And actually, this makes sense, as if we think about the feature layers, we have the two main categories of minimap and screen, remember? This Move_screen command is likely the "move" command on the current "screen". If that's true there should be a Move_minimap as well. Let's see:

python3 -m pysc2.bin.valid_actions | grep "Move_minimap"
20/Scan_Move_minimap                                  (3/queued [2]; 1/minimap [64, 64])
332/Move_minimap                                       (3/queued [2]; 1/minimap [64, 64])

Hypothesis correct, yay! Let's move on.

_ATTACK_SCREEN = actions.FUNCTIONS.Attack_screen.id

The Move_screen command has the advantage of being used as an example in Deepminds documentation. Attack_screen is not. But the parameters look the same as Move_screen:

12/Attack_screen               (3/queued [2]; 0/screen [84, 84])

So, this should be the command to attack somewhere on the screen. In StarCraft II, if you press attack and then click a unit, that unit will be attacked. Quite obvious, I know. However, if you click attack and then somewhere on the screen, your unit(s) will go towards that point and attack anything they encounter on the way.

_SELECT_ARMY = actions.FUNCTIONS.select_army.id

Since you can have both army units and worker units in SC2, this command selects all army units which is useful when you want to consolidate all your army units or cannot select an area on the screen with the mouse without selecting workers. It only has one parameter:

7/select_army                 (7/select_add [2])

I had to look through pysc2.lib.actions to figure out what this was exactly, but it's a boolean denoting whether to add all army to current selection (True) or select all army units regardless of what is currently selected (False). So if you have a few workers selected and use select_army with the True flag, you'd have all army units and those workers selected.

_NOT_QUEUED = [0]

Here we create a variable for the parameter of the current action not being queued. As previously stated, I'm not sure how this queue works. In pysc2.lib.actions we find that:

"""queued: Whether the action should be done now or later."""

and

queued=ArgumentType.enum([False, True]),  # (now vs add to queue)

So, 0 (or False) as assigned to _NOT_QUEUED means perform the action now, while True (1) means add to the queue. It could be that actions in the queue are automatically performed if the agent class step function is called without an action being returned. Could also be that actions in the queue are performed the next time the agent sends a no_op. And another possibility is that this queue is only active during the same step, so that you can put a bunch of actions together and send them. Guess we need to experiment to know more, and I'll just move on for now because I don't think we need to worry too much about it at this point.

_SELECT_ALL = [0]

The final constant defined is the parameter for the above mentioned _SELECT_ARMY action. _SELECT_ALL is not a great name, as 0 (False) means select all army units without adding current selection. Then again maybe the variable name works fine, if you read it as "select all army". Anyway. Now it's finally time for the actual class MoveToBeacon!

If you want, take a breath and go pee or something. We got this far, now prepare for the real stuff.

class MoveToBeacon(base_agent.BaseAgent):
  """An agent specifically for solving the MoveToBeacon map."""

The class definition. We inherit from the BaseAgent class. Not much more to say.

def step(self, obs):
  super(MoveToBeacon, self).step(obs)

We need to define the step function, as this is where all of our bot's behavior happens. It gets the obs variable as input which is the variable containing all of the observations for that step. Then we call the same function in the parent class (BaseAgent) using super. Why do we call the parent function? Well, if you remember from before (and if you don't just scroll up and look) the BaseAgent increases number of steps and reward every step. Using super here we can do that without explicitly writing those two lines ourselves. You can read more about super on stackoverflow.

Let's look now at the entire rest of the code block and then break it down:

if _MOVE_SCREEN in obs.observation["available_actions"]:
  player_relative = obs.observation["screen"][_PLAYER_RELATIVE]
  neutral_y, neutral_x = (player_relative == _PLAYER_NEUTRAL).nonzero()
  if not neutral_y.any():
    return actions.FunctionCall(_NO_OP, [])
  target = [int(neutral_x.mean()), int(neutral_y.mean())]
  return actions.FunctionCall(_MOVE_SCREEN, [_NOT_QUEUED, target])
else:
  return actions.FunctionCall(_SELECT_ARMY, [_SELECT_ALL])

There's an outer if/else here, if _MOVE_SCREEN in obs.observation["available_actions"]. The "available_actions" contains, well, all available actions at this current observational step. If no units are selected, the move command will not be available. Makes sense, right? So if the move command is not available, we have no selected units, hence the else: return actions.FunctionCall(_SELECT_ARMY, [_SELECT_ALL]) as that will select all your army units. In the current context, the MoveToBeacon mini game, that means our lonely marine.

Now, if we have our marine selected, we want to move it to the beacon. How do we find this beacon?

player_relative = obs.observation["screen"][_PLAYER_RELATIVE]
neutral_y, neutral_x = (player_relative == _PLAYER_NEUTRAL).nonzero()

We start on the first line here by getting the (84, 84) matrix for _PLAYER_RELATIVE. If you remember from before, that will show 0 for background, 1 for where the marine is (self) and 3 where the beacon is (neutral). The next line gives us all the points in the (84, 84) matrix that are equal to 3 (equal to _PLAYER_NEUTRAL) and that are not zero. As an example, consider the following small matrix:

matrix = numpy.array([
[0, 1, 0, 0],
[0, 0, 3, 3],
[0, 0, 3, 3],
[0, 0, 0, 0]
])

(if you have your terminal ready you can open the python interactive prompt or use ipython to follow along and experiment for yourself if you're unsure of how this works)

Imagine this 4x4 matrix is the game screen, instead of the usual 84x84 matrix we're actually working with. Each number in this 4x4 matrix is a pixel. And we see that the beacon covers a total of 4 pixels, the pixels saying 3. The marine covers one pixel in the upper left, that's the 1. If we use nonzero on this we get:

In: matrix.nonzero()
Out: (array([0, 1, 1, 2, 2]), array([1, 2, 3, 2, 3]))

We get two arrays for the positions of nonzero values. The left array contains the row values and the right array contains the column values. So the marine for example is at position (0, 1) and can be accessed directly from matrix like this:

In: matrix[0][1]
Out: 1

But we only want the beacon, we don't care about where the marine is. So we can do:

In: numpy.nonzero(matrix == 3)
Out: (array([1, 1, 2, 2]), array([2, 3, 2, 3]))

In: (matrix == 3).nonzero()
Out: (array([1, 1, 2, 2]), array([2, 3, 2, 3]))

Now we get only the pixels where the beacon is! So to get the "upper-left" 3 we can do:

In: matrix[1][2]

In the main code above, they've said:

neutral_y, neutral_x = (player_relative == _PLAYER_NEUTRAL).nonzero()

Why y before x? Because usually if you look at most graphs, you put x on the horizontal line and y on the vertical. Since the left array are rows and the right array columns, by doing this we can think better since it'll be like any regular graph. Or something.

Get it? Good. Otherwise, play around on your own with these functions, it helps when trying to get stuff like this to experiment you know.

Now that we've looked for where the beacon is, there's a test in case there's no beacon found:

if not neutral_y.any():
  return actions.FunctionCall(_NO_OP, [])

If no beacon was found on the screen - if there are no 3s in the matrix - we send back a no_op action.

If the neutral_y did contain at least one 3 then we move onto the final two lines of code:

target = [int(neutral_x.mean()), int(neutral_y.mean())]
return actions.FunctionCall(_MOVE_SCREEN, [_NOT_QUEUED, target])

The target is of course the beacon, and here we find the middle of that beacon by taking the mean value of the x and y coordinates where 3s were found. In the 84x84 matrix, the beacon covers around a 10x10 area I think.

Finally, we return the action _MOVE_SCREEN with the parameters _NOT_QUEUED and the coordinates target.

That's it! Now we know how the scripted agent MoveToBeacon works. More or less :)

In the next post/chapter, we shall use what we learned to create an agent that learns how to solve this game without us specifying how it should do that. With the assumption that we basically only need the same few actions and observations that the MoveToBeacon scripted agent uses, we will build or own bot using reinforcement learning. Maybe this assumption is wrong and we need more stuff, but whatever, we'll just hack that if/when we get there!

Stay tuned for the next episode!

why you should use duckduckgo

foh@gamescapad.es (gamescapades) — Sun, 03 Sep 2017 02:00:00 +0200

see, they tell you it's about privacy. and sure, a search engine that doesn't track everything you do and cares about your privacy is great. but if you've ever used it you know that the results are not always great.

but you know what i just realized? duckduckgo is better in some ways because it doesnt always give you what you're looking for. at this point, you can likely press “i’m lucky” in google because it’s so personalized that it knows what you mean, even before you know you need it.

not the case with duckduckgo. since the results are not always what you want, you may have to alter your query a few times. but if you look at the results, you can find weird things you find interesting that you wouldn’t have found otherwise. they may even give you new perspectives on things.

the old interwebs was better because this was the experience everyone had. now we're all in our own information bubbles, never seeing anything unrelated to what we want.

but you can still get a feel for how we used to search, thanks to duckduckgo.

gamescapades

reviews

Mirror's Edge Catalyst is a Crap Game

home

essays

Increasing Diversity Decreases Team Blind Spots

Planet Ghibli Part 3 - How AI can increase inequality in society if we do not engage in diversity

contact

about

"not totally horrendous"

Increasing Diversity Decreases Team Blind Spots

Holistic Sustainability - Automating subtitles for Swedish public service with OpenAI Whisper

The result

Thoughts and next steps

Planet Ghibli Part 3 - How AI can increase inequality in society if we do not engage in diversity

Why shouldn't we trust automated decision making?

Shifting the power

So what can we do?

Planet Ghibli Part 2 - Making a Video

Quick links for this post (a.k.a. TLDR)

Before we move on

Training models briefly explained

Actually training our model(s)

Extracting frames and running the style transfer

Style transfer

Putting the pieces back together

Discussion

Conclusions and up next

Planet Ghibli Part 1 - Setup and Play

Very brief explanation of style transfer that is likely to annoy people fluent in mathematical details

Let's get this stylin' goin'

Access apps like Jupyter notebook remotely using SSH tunnels

Contents

Some prerequisites

Running an app locally

Accessing an app remotely

Multiple tunnels

Multiple tunnels: step 1

Multiple tunnels: step 2

Summary and more possibilities

TLDR

Troubleshooting

Creating Websites Like It's 1999

When I was seventeeeeeeeen

ps

Mirror's Edge Catalyst is a Crap Game

Building Bots in StarCraft 2 for Psychologists

Introduction: humans are amazing

pysc2 - a love story between Blizzard and Deepmind

Installation

Overview of the framework

Observations

Actions

Putting it together

Step 1: understanding the scripted bots

Case study 1: class BaseAgent

Case Study 2: class MoveToBeacon

why you should use duckduckgo