The Sledgehammer – Version 2.0

January 11, 2014

What Software Testers Can Learn From Video Game Speedrunners

Filed under: Games, Quality Assurance — Tags: , , — Brian Lutz @ 2:53 pm

I see a lot of this when I attempt to play video games.

Over the course of the past week, I’ve spent far more time than I care to admit watching other people play video games far better than I could possibly do it.  Every year around this time, Speed Demos Archive puts on an event known as Awesome Games Done Quick, where a group of speedrunners gets together and plays games nonstop as fast as they possibly can for an entire week, streaming it online as a fundraiser for the Prevent Cancer Foundation (and a pretty successful one too, raising nearly $450,000 last year, and as of this writing the total for AGDQ 2014 is sitting at roughly $663,000 with about a day to go, plus whatever bonus streams follow the main event.)    For someone such as myself who has pretty much no skill whatsoever when it comes to anything requiring fast twitch reflexes, it is fascinating to watch this type of thing for several reasons.  First of all, the amount of skill being put on display by the various speedrunners is amazing.  And the second (and perhaps more compelling) reason is that as the various speedrunners go through their runs, they tend to provide a running commentary explaining what they’re doing as they go along.  And quite a bit of what they’re doing is, quite frankly, breaking the games.

But as I’ve watched the marathon and seen the types of techniques that speedrunners use, it has occurred to me that there are actually some things I can learn in my professional career as a software QA engineer from watching this type of thing.  Even though I don’t do anything related to games in my job (and only one or two things I have ever done in my career have come even remotely close to it) it seems to me that a lot of what of people do in the course of speedrunning games is quite similar to what I do in testing software, with one significant difference:  As a tester, I’m trying to find problems to get them fixed, speedrunners are typically trying to find them to completely break things.  And to be perfectly honest, I think the speedrunners might be winning on this one, judging from some of the ways they can take tiny little glitches and completely break entire games with them.  In most cases this has no real impact other than to beat games far more quickly than they were ever intended to be beaten, but we’re generally talking about twenty year old games here.  If you’re running mission-critical software in an enterprise environment and things like this are happening, you might find the impact of something like this to be far more problematic.  Naturally, it’s best to catch these types of things well before the software (be it a game or something more functional) goes out into the wild.  As such, I thought I’d put together a post that goes through some of the lessons that I have learned from watching speedrunners during AGDQ.

1. People will go to great lengths to make even trivial gains in performance.  Although the speedruns in AGDQ are compelling enough on their own, the part that really makes it interesting is the commentary that goes along with most of the runs.  Whether it’s coming from the speedrunner(s) playing the game or from providing a play-by-play from the couch, it quickly becomes clear that the people doing this stuff have put as much thought and effort into this as most people would put into far more serious subjects.  I suspect that the collective knowledge that has been gleaned from one of the more popular speedrunning games such as Super Mario Bros. could fill a book, or at least an article in an academic journal.  Nonetheless, even for games that have been well documented and well understood for years, people are still trying to find ways to shave fractions of seconds off their times.  In particular, one of the popular (yet somewhat controversial) categories in speed running tool-assisted speedrunning, also known as TAS.  Tool-assisted speedrunners use various tools to do things like run games a single frame at a time and use savestates to keep running through segments until they can figure out the optimal paths through or pull off difficult tricks, which allows them to eventually work toward what could be considered a fully optimized run.  In many cases, these optimized runs can be much faster (often by multiple minutes) than what even the best human players can manage, but they also tend to do this by using tricks that human players would not be able to do.  Nonetheless, the TAS players can find hidden strategies that can save time in regular speedruns, but at the same time can also be very difficult and/or risky.  It’s not uncommon to see speedrunners taking big risks on difficult tricks that might save them a fraction of a second if they pull it off, but can cost them much more than that if they don’t.  Speedrunning is by its very nature competitive, and at times it can be mere fractions of a second that can separate players in a racing each other on an hour-long speedrun (I don’t have a way to link to it yet, but the 4-way Super Metroid race from AGDQ 2014 is a very good illustration of this.)

Although this isn’t a scenario that necessarily translates to real-world software in the same manner (as you might imagine, when working with most types of hardware and software the goal is far more to reduce risk as much as possible than to reward it,) one thing I do typically see in the course of my daily workflow as a tester is that there are a lot of repetitive tasks that come up, not just in the actual testing, but in the course of dealing with the other associated tasks that come along with it such as bug tracking, test case management, setting up test environments and reporting results.  Although the use of automation in test case execution is widespread and can save significant time over manual testing in situations where it can be applied, I’m not dealing with much of it in my current job.  Nonetheless, even if you’re not automating  your test cases, you can probably identify little repetitive tasks here and there that you might be able to automate with something like a batch script or a macro.  Even little things that don’t seem like much can add up over time, and in the long run you can make significant performance gains out of little things.

2. Things that may seem random rarely are.  As you watch the various speedrunners going through their runs, one of the things they point out frequently is where things are or aren’t random in the games.  As you watch the various runs, you realize that at least under specific conditions, most seemingly random things aren’t actually random.  This is frequently important because a lot of the strategies (speedrunners typically call them “Strats”) depend on certain things happening at certain times.  On the flip side of the coin, random events tend to be a hindrance, as they can interfere with things often.  Mostly through exploration using TAS and other playthroughs of the game, it is possible for them to determine what is going to happen when, and also to figure out ways to precisely control the circumstances in which certain things happen and manipulate them to their advantage.  While testing software, often one of the biggest challenges testers face is trying to come up with consistently reproducible scenarios for bugs that have been reported because you have no way to verify if you actually fixed a bug if you don’t have a reliable way to get that bug to manifest itself in the first place.  This can be difficult, especially for bugs that may have been seen only once or twice, or issues that have been reported by non-technical users who provide only limited information and in a production environment where you might not have access to the debugging tools you’re used to having on your test bench.  It is for this reason that you need to be familiar with the environment you’re working in, and that you know what circumstances might lead to one particular code path instead of another.  If possible, you also want to have ways to collect at least some sort of data from low-information users in situations like this.  In many cases, understanding what circumstances might cause certain unwanted behaviors to occur in a piece of software can be largely a matter of determining the state of the environment at the time the problem happened.  Granted, this can require going rather deep into things, but speedrunners (and especially TAS runners) have gone surprisingly deep into the games they’re speedrunning, and have managed to do some rather surprising things, as this tool-assisted run of Super Mario World from AGDQ2014 illustrates.  It starts out unusual, gets downright weird, and goes…  Well, you’ll just have to watch.

3. You’re always going to miss something no matter how much testing you do.  The vast majority of games being played in the AGDQ marathon were some of the best-selling and best known games of the time when they were created.  Although the tools and services available today to game developers has allowed many smaller indie developers to put out products that can rival the big-name studios, in general a lot of games being shown were produced by rather large teams of developers, testers, artists and other support staff, often across multiple companies.  That means that by the time these products made it to the store shelves back in the day (something that has, ironically, become less and less of a reliable indicator of a product’s quality as console technology has reached the point where patching has become not only possible but practically expected)  they may have had hundreds of people involved along the way, including large numbers of testers dedicated to finding and reporting bugs to be fixed.  In spite of all that, the speedrunners still manage to find glitches, exploits and other bugs.  Not all of these are necessarily going to be useful for reducing speedrun time (in fact, a lot of these don’t do much more than crash things.) but these can be little things, big things, or somewhere in between.

Of course, very few of these glitches are things that a player going through the course of the game in the intended manner would ever run into (a lot of them involve finding ways into areas that the player is not supposed to be able to go into,) but unless they’re specifically restricting themselves to this, most speedrunners are going to use every glitch they can manage to get.  And I’m sure that there are developers and testers out there who have smacked themselves in the head after seeing some of the stuff that the speedrunners have pulled off in their stuff.  In the course of running a test pass on a game like the ones featured here, a lot of the scenarios where the glitches appear would be considered edge cases, which are things that very few users would even go anywhere near.  The main problem with these edge cases is that you’re generally wandering well off the “happy path” that normal users would be on, and in general the returns on these test scenarios tend to be very low in terms of the amount of time spent running them.  Then again, if you aren’t going to find the problems here, there’s a very good chance that someone else will gladly find the problems for you.  And you’re probably not going to like the results when they do.

4. Anything that is deemed unnecessary will be skipped one way or another.  During this year’s AGDQ, one of the featured runs was for Resident Evil 4, a game that I’ve never played (it’s not the type of genre I’m interested in) but which was still quite interesting to watch.  One of the biggest things I took away from this particular speedrun was that the player basically just ran right by probably 75% of the enemies in the game without a second thought, and suffered no ill consequences for doing so.  A lot of these fights would likely be rather difficult (and time-consuming) if the player was to actually do them the way the developers intended, but oftentimes it turns out to be completely unnecessary, as they just run right by and keep going.  Of course, in a speedrun saving as much time as possible wherever possible is crucial, so a lot of effort goes into cutting out even trivial things.  In particular, cutscenes and dialog are frequent targets of speedrunners, who will often take rather unusual steps to keep them from happening or exit them as quickly as possible.  In some cases, you’ll see people literally reset the game or quit out and reload in the middle of a speedrun, because starting from scratch and reloading from a save can in some cases be much quicker than watching a cutscene.  As long as the established ground rules for a particular game allow it, this is considered perfectly normal.

Another thing you see that happens quite a bit is that players will intentionally take damage in a lot of instances in order to use the temporary invincibility that typically goes with it  to bypass things.  In games, people tend to think of health or energy (or even lives) as something they have to try to keep as much of as possible, but speedrunners tend to treat these things primarily as a tool.  In particular, games like F-Zero GX (one of the most notoriously difficult games in recent memory, and one of the major highlights of the past couple of AGDQs as speedrunners have absolutely destroyed it) give you an energy bar that acts as both your health meter and something that can be consumed as a boost, allowing you to go faster but significantly increasing your risk of failure by doing so.  Then again, this is a normal (and expected) mechanic of this particular game, but taking intentional damage to bypass obstacles and improve speed is surprisingly common in many speedruns, especially for 8-bit games like the Mega Man and Ninja Gaiden series.  In some games, strategically placed intentional deaths are a common occurrence as well if some advantage can be obtained by doing so.  Then again, most players need to use that health and those lives just to keep themselves from hitting the game over screen too soon, so a lot of this comes down to having enough skill in the first place to avoid unintentional damage as much as possible, since it becomes a lot riskier when people play this way.  This means that in addition to all the various strategies and optimizations involved in the whole process, there’s also quite a bit of raw skill required just to even be able to think about speedrunning a game (of course, even if you can’t do live speedruns you can always try to do TAS, but that’s basically something entirely different.)

5. If there’s a way to pull things off the rails, someone will find it.  In many ways, this really ties into #4, but I feel it should also be considered separately.  One of the most popular genres of games for speedrunners is the so-called “Metroidvania” games (of which the 2D Metroid and the non-linear Castlevania games such as Symphony of the Night are the most prominent examples,) which typically are played on large non-linear maps but ultimately still have a linear progression that the user is expected to follow.  Of course, it is possible to follow this linear progression and do a speedrun that way, but most of the time the goal is to finish things as quickly as possible no matter how this is accomplished, so when a new game of this genre comes out, the first thing the speedrunners do is try to find so-called “sequence breaks,” which are strategies that allow the player to subvert the expected linear progression of the game and skip significant portions of the game entirely and acquire items that they aren’t expected to have  until much later in the game.  Of course, it’s gotten to the point that a lot of developers these days just hide intentional sequence breaks into the games, but in most cases these have come about as a result of players messing around with things they aren’t supposed to be messing around with, trying to actively subvert the intended order of the game. 

The effort that goes into testing a particular piece of software is as much a matter of planning as it is execution.  After all, you (generally) have a specific set of requirements that the software must be able to meet, and you need to be able to demonstrate that the software can meet those requirements.  And these days more than ever, security testing becomes a very important part of those test plans.  After all, no matter what type of software you are working with, it’s highly likely that someone out there will be trying to find ways to get around whatever limitations happen to be in it, especially if you’re dealing with any system that stores sensitive data.  But even aside from that, you can find yourself surprised by some of the things you’ll see users try to do with your software, things you would never expect.  As you go through the various test passes and validations that you might do over the course of a software development life cycle, you start to develop a surprisingly deep understanding of how things tend to work in the system, even if you aren’t working directly with the code.  As a result, you tend to build a bit if an intuition for some of the unusual things users might try somewhere along the line.  Don’t hesitate to try some of these things out; you never know just what kind of weird issues you might manage to run into.  Not that all of it will necessarily get fixed (after all, developers’ time is a finite resource, and you eventually have to ship something) but if you can think of it, chances are that at some point someone else will do the same.

All things considered, there’s actually a surprisingly large correlation in the methods used by speedrunners and software testers for their respective tasks.  In both cases, people are going deep into the inner workings of the software they’re using to try to find things that don’t work the way they’re supposed to.  Both use a lot of the same methods, and both find a lot of the same issues.  It’s how these issues are used where things tend to diverge though.  As a tester, it’s naturally your job to find these issues in order to get them fixed.  As a speedrunner, you’re trying to find issues that you can use to break things even further.  Either way, the results can be fascinating to watch.

March 11, 2012

The Basics of Software Testing: How Well Does Your Software Really Get Tested?

Filed under: Quality Assurance, Technology — Tags: , — Brian Lutz @ 1:14 am

(Note:  This post is part of what I plan to turn into an ongoing series of posts describing some of the basics of software testing.  Although there has been a lot written on the subject of testing over the years, I have found that a lot of it is, to put it nicely, not particularly user-friendly.  Boldly ignoring the adage I saw on a recruiting poster at Microsoft once saying you should work for a particular team because you never want your Mother to understand what you do for a living, what I intend to do here is explain, in a nutshell, how software gets tested, how the results of that testing get dealt with, and generally what the role of a software tester is in the development process.  I’m trying to keep the jargon to a minimum here, but I apologize in advance if this gets just a bit too technical.)

In our modern society, it can be very easy to take things for granted.  Take, for example, your household appliances.  Most of the appliances found in a typical modern household are items that just a hundred years ago were either years from being invented at all, or very expensive and difficult to attain for most people, with the tasks they perform requiring significant quantities of manual labor to perform.  In particular, it could be an all-day project just to do the laundry by hand in days of yore before washing machines were invented.  In our modern age, this once laborious task has been reduced down to throwing your clothes in a machine with some detergent and maybe a bit of bleach, setting a dial and walking away and throwing it in another machine when the buzzer goes off at the end of the cycle.  In fact, about the only time most people are going to give much thought to the process of doing laundry is when something goes wrong, such as when the machine breaks down for one reason or another, or when an imbalanced load creates a vibration in the machine that can shake the entire house.  In the same vein, the previous apartment I lived in before I moved to Downtown Bellevue provided me on a couple of occasions with firsthand experience with the joys of a hot water heater developing a leak at 8:30 on a Sunday evening and soaking the surrounding carpeting.  Can’t say I ever paid much attention to it before that one happened.

By the same token, much of the software that runs not only our computers but also an ever-increasing number of other electronic devices found in a modern household tends to fade into the background for most users 99% of the time, and only really gets noticed when something goes wrong or proves incapable of doing whatever that the user is trying to use it for.  When things start breaking, it’s easy for the average user to jump to conclusions and blame a lack of testing on the part of the developers for their present woes.  Although in some cases that assertion may not be entirely unjustified, it’s often a lot more complicated than that.  It’s been said that there is no such thing as a completely bug-free piece of software (with the possible exception of one of the “Hello World” programs that you tend to find on the first page of a beginning programming book,) and as programs continue to get more and more complex, there’s more and more things that can possibly go wrong.  Naturally, this also means that more and more things will, in fact, go wrong at some point.  And by the time all the development and testing is done and the software goes out the door, it’s inevitable that a few things will have slipped through the cracks, no matter how much testing gets done and how many testers you have hammering on it. 

What might be a bit less obvious to the average user is that there will also be a number of bugs in any product that were identified and filed by the testers, but were not fixed in the final product.  Some people might find it shocking that software would be shipped with the full knowledge that there are bugs in it, but it’s really just a natural part of the process of developing software, and 99% of the time, it’s nothing you would ever notice anyway.  I’ll most likely save the subject of how bugs are found and how they’re dealt with for another post, but when it all boils down, it mostly boils down to prioritizing things and making sure that the limited development resources on a project are being used in the most efficient manner possible, and at times a bug fix will prove to be more trouble than it’s worth.  For a novice tester this can be a difficult concept to understand at first, but over time they will learn to pick their battles and know what they might need to push back on and what they should just let go.  In short, not all bugs are created equal.  In a future post in this series, I plan to go into a bit more detail on how bugs are handled within the software development process, but in general, there’s some process of sorting and prioritization (commonly known as triage) that takes care of this aspect of things.

That said, when a product (or in most cases, a smaller portion of a much larger product) gets handed off from the developers to the QA team, how does it get tested?  Although a lot of the details are going to be dependent on the specific item in question, ultimately the approach is going to be similar no matter what you’re testing.  It is for this reason that a favorite technique of technical interviewers interviewing candidates for test positions is to specify some random item, and ask the candidate how they would test it.  Although most of the time this ends up being some sort of software component, there’s no reason this has to be limited to software, and on a number of occasions I’ve been asked how I would test completely random items ranging from vending machines to those misting systems they use to spray water  on the produce at your friendly neighborhood grocery store.  You’ll find in an interview for a technical position like this that the interviewers are looking less for “correct” answers to this problem than they are looking at the thought process that goes into it.  For even the most mundane and utilitarian of items, it should be possible (and expected) for an experienced tester to quickly devise a plan for testing many different aspects of that item, including functionality, performance, stress, real-world usage, globalization, and a number of other situations that particular item may be involved in.  Although the product in question is often something that has little to no relevance to the product that the tester is ultimately going to end up working with, the methods will be similar, so this still makes a good illustrative example.

When it comes to actually testing software, the approach here is going to be similar but at the same time somewhat different, and in my experience tends to be largely dependent on what is being tested, and where in the development process you happen to be joining.  For example, if you’re joining a project that has been in development for quite a while and which already has a test team in place, chances are that a lot of the groundwork of test plans, test methods and test cases will already be in place, and the duties of the tester will be largely in maintaining the existing base of test cases, running test passes, and (if applicable) working toward automation of the existing cases.  On the other hand, if a tester is joining in on a project that has had little to no testing done on it prior to his or her arrival, chances are that the tester is going to be doing a lot more of the foundational work of creating the test plans and the test cases on which the future testing will be based.  Not all the testing will necessarily be this structured though.  As with above, I intend to go into more details on how test plans and test cases are created in a future post.

Another common type of testing that takes place is known as ad hoc testing, which is basically the “bang on it with a hammer and see what breaks” type of testing that most ordinary people will think of first when the subject of testing software  comes to mind.  This can either be directed, where the tester is focusing on certain areas or certain features but still isn’t working from formal test cases, or undirected, where the tester is basically just messing around with things to see what breaks.  Even on a project with well established test plans, test cases and automation in place, there is still benefit to be gained from this less directed form of testing.  In my own experience, I have found that a good session of ad hoc is not only a surprisingly effective way to get started on something, as not only does it frequently catch a number of bugs and give the developers something to work on fixing while I’m in the process of establishing more formal tests, it’s also a good way to become familiar with the product and start planning for more formalized testing later on in the process. 

Although ad hoc tests can be a good way of  identifying a decent quantity of bugs quickly, they do come with some disadvantages.  For one thing, an experienced tester might be tempted to dig into deep dark corners of the product and try to find some really arcane bugs.  Scenarios like this are what’s known as edge cases, and represent the types of situations that are going to be encountered rarely (if ever) when real-world users start using the product.  Fixing these more arcane bugs will generally be a low priority for the developers, as they will often have far more important things to be working on.  Even so, the benefits of ad hoc tests tend to be worth the effort that goes into them, and on large software projects it’s not uncommon for what’s known as a “Bug Bash” to be scheduled somewhere in the development process.  A Bug Bash is basically a period of time set aside where developers, testers and even project managers all get together and just bash on the product (with varying degrees of direction, usually more generalized than specific) to try to dig up as many bugs as possible in a certain period of time.  Some places even go so far as to turn this Bug Bash into a competition with prizes for whoever finds the most or the biggest bugs.  This can be a useful tool for getting different sets of eyes on the product, and perhaps getting a bit out of the tunnel vision that testers can sometimes develop in the course of testing things.

Of course, a piece of software in development is a moving target for testers, and as a result, it is a necessary part of the development process to run the same sets of tests frequently to make sure that not only are things working the way they’re supposed to as they get developed, but also to make sure that further development on other areas of the product isn’t breaking existing functionality or introducing new bugs into existing components.  As a tester, running the same sets of tests over and over repeatedly tends to become incredibly tedious.  This is where automation comes into the picture.  Automation is another subject that will most likely get its own post later on, but in a nutshell, automation is the process of using a piece of software known as a test harness to automatically control the functions of the software being tested.  Since the vast majority of test cases that will be run in the process of testing a product are going to be specific sets of actions with  predictable expected results, this means that the test harness can be programmed to perform these actions automatically and verify that the resulting output of those actions matches what is expected, assigning a pass or fail result to each case based on this.  Having the software being able to essentially test itself can greatly reduce the drudgery involved in running the repeated test passes necessary in development, and is especially useful for the set of high-level tests that gets run for each build to make sure that it is acceptable for testing (known as Build Verification Tests, or BVTs.)  Although automation can greatly reduce the workload of the testers, there are limitations to what can be tested with automation.  For example, user interfaces tend to be a lot more difficult to automate compared to lower-level components like APIs and webservices, and it is virtually impossible to fully automate the testing of a piece of software because of the sheer complexity involved in most software projects these days. 

There are a number of off-the-shelf products on the market that can be used to create automated tests, but in my experience I’ve found that a lot of the places I’ve worked tend to reinvent the proverbial wheel when it comes to test automation, and it’s surprisingly common to see automation for a product being handled by a proprietary test harness made specifically for that product.  Because of this, the process of automating tests can require far more programming than testing.  Most test automation is created by a group of engineers known as Software Development Engineers in Test (commonly abbreviated to SDETs) who tend to exist in a bit of a fuzzy space somewhere between the development team and the test team (usually closer to test than development though,) and who spend a majority of their time programming automation tools and creating the automated tests that will be used.  In fact, it is not uncommon to find development teams where all the testers are SDETs, and comprehensive knowledge of software development and programming languages is an expected prerequisite to qualify for the work in addition to the usual test experience.  This isn’t to say that any project manager in their right mind is going to rely exclusively on automated testing to verify the quality of their product; in most cases this means that you have people who are wearing both the developer and the tester hats as needed.  There are quite a few other types of tests that take place in a software development cycle (such as performance testing, stress testing, internationalization testing, unit testing and others,) but the ones you see above tend to account for the vast majority of testing that happens on a product under development. 

As discussed above, people tend never to notice all the work that goes into testing a piece of software until something goes awry, at which point they invariably suspect that no matter how much actual testing took place, there obviously wasn’t enough of it.  And yet in spite of this perception, testing is a fully integrated part of the software development process, just as important to the success of a product as the developers who do the actual development work are.  As I continue with this series of posts and try to explain in more detail how software gets tested, I hope to give a little bit more visibility to the role of testers and just how important the work they do is in ensuring that the average user rarely has occasion to pay much attention to the software they are working with.  It’s a nerdy job, but someone’s gotta’ do it…

January 22, 2010

A Bit of Temporary Madness

Filed under: Random Stuff, Seattle, Technology — Tags: — Brian Lutz @ 12:33 am

Although for the most part over the last decade or so I’ve been blessed with (mostly) steady employment and the means to achieve and maintan self-sufficiency, when I go back and think of all the various places where I’ve been throughout my career, it always seems to be the shortest jobs that end up being the most interesting ones, at least in terms of the stories that come out of them.  Of course, “interesting” is rarely the first thing to come to mind when you’re manually mapping out all the ports on a PBX system by running your hands through a snake pit of wires or sitting in the middle of an unheated warehouse in January running the same imaging script on 400 computers, but most of the standard-issue 9-to-5 jobs I’ve worked over the years have produced few memorable moments.  After all, I suspect that giant software companies rarely pay people in unmarked envelopes of $1 bills out of the Pepsi machines.  There are always exceptions to this rule (a few jobs that were either particularly interesting or particularly horrible do stand out) but as odd as it sounds, some of the jobs I spent the least time in are the ones I remember most.  And I suspect that the one I’m at right now is no exception. 

This past week, I have found myself in a third-floor loft in Seattle’s Pioneer Square neighborhood along Yesler Way, the prototypical Skid Row from which the term originated.  I am currently helping out at a small startup for a couple of weeks as a tester for an iPhone app, although in an office of five people that rather vague description does meet up with a fair bit of scope creep.  With my interest in urban archeology and old stuff, it’s kind of interesting to get a close look at some of these historic buildings.  As most people know, Pioneer Square is where many of the oldest buildings in Seattle are found, with many of the buildings dating as far back as 1889(much of Seattle’s central business district burned down in the Great Seattle Fire of 1889, and a lot of what’s here now was built to replace the burned down buildings) and having seen the Alaska Gold Rush of the late 1890s. 

Stairs:  They sure don’t make them like they used to, and there’s a good reason for that.  I could see someone tumbling headfirst down these stairs to an untimely demise quite easily, and hesitate to try to traverse them without the handrail.  It’s the kind of thing one might regard as adding character to the building, and if that’s the case, there’s no shortage of character in this place.  I’m not trying to say that the place is old and decrepit, but I suspect that it’s seen better days… back in 1905. 

The holes in the floor which allow one to see through to the floor below add a nice touch though (unless, of course, you’re the people down below, but that’s another story.)  Aside from this little startup I’m with, I think half the building is occupied by various Yoga and Pilates studios, with some random garage band using the floor above ours as their practice studio, just for good measure. 

At least when I look out the window I can see the Sound.  At least I can if I manage to look past the much maligned Alaskan Way Viaduct and the barren alley below.  There’s actually a number of ghost ads and signs on the building you see here, but most are too faded to make out much more than the faintest of details.  I suspect the graffiti is of comparatively recent origin, but even that seems to have been around for quite a while by now.

To be honest, there’s a bit of a culture shock that comes with having spent so long in the highly bureaucratic environment of a major software company with at least 30 or 40 people (and usually more) working on any given product and then going to a startup with less than 10 people in the whole company.  After having spent so long as a relatively minor member of some rather large test teams, going to a place like this and finding that I am pretty much the whole QA department (for a while anyway) is a little odd. For one thing, it means that I pretty much have to start from scratch when it comes to testing the product and setting up whatever cases and frameworks need to be used.  This isn’t necessarily a bad thing.  As a tester, there are few times when you feel better about your job than when you get a chance to pick up a new, largely untested program for the first time and go to town finding and filing all the bugs.  Back when I was doing international testing, and got a hold of the first pseudolocalized* build of the rather large piece of enterprise software I was working on at the time, I think I managed to open 100 bugs in one day.  Some people would think that successfully shipping a product would be the time when a software tester would be happiest with their work, but the problem with that is that as no product is perfect, and on just about everything shipped product I’ve ever worked on, I’ve been able to name at least five significant bugs that didn’t get fixed, either for being discovered too late to do much about them or for just not being significant enough to warrant fixing.  It’s a bit of a curse, but you get used to it, and you at least have a reasonably good idea that at least all of the major stuff is working like it should be.  Of course, when you have to go through half a dozen project managers and dev/test leads (collectively, it’s not THAT bureaucratic most of the time) to get approval to have even the most trivial of bugs fixed, you find it hard to be surprised when half the bugs you find get punted back to you.  And don’t even ask what it takes if you try to actually change something in the product.

Of course, the small-team environment does come with its advantages too.  For example, it provides a chance to to something besides sit around and run tests all day.  I’ve even managed to find myself involved actively in the design of various aspects of the product, and generally a lot more involved with it than I would have ever been in some of the big teams I’ve been on previously.  I’m sure I’ll have more details on this once the product I’m working on gets completed, but for now I’ll just say that I’m working on something that’s just about the last thing I would have ever expected to find myself working on.  I also suspect I’ll probably nor exactly be bragging to my mother about this one, but that’s another story.  Oh well, at least it keeps things interesting, right?

*Pseudolocalization is the process used early in the localization of a product to test the ability of the product to be translated into other languages.  This is done byautomatically replacing all of the strings within the product with dummy text that looks similar enough to English to still be readable, but is made of foreign characters.  In addition to this, most of the time the pseudolocalized strings include additional padding characters to show where text strings in the user interface may not be long enough to accomodate their foreign equivalents.  For example, if you have a string like “Never eat grasshoppers while they’re still hopping,” the pseudolocalized version might look something like “[Ñĕɤȩȓ £æƭ ƍ®äšśĥضÞëƦ§ Ŵħìﺄє ҭђӘч`ʁȩ șțɨȴĺ ĥơҏp!ӆϑ!!! !!! !!! !!! !!!]” inside the product.  Sure it looks silly, but you should still be able to read it, and this can tell you a number of things.  For example, if you’re seeing big white boxes in the middle of that, there are characters that either aren’t supported or can’t be handled properly by the product.  If you’re missing the end bracket, then you’re likely to run into truncation problems when you try to put the foreign equivalent to the string in there.  If you’re seeing the string just like it was in the regular version, you’ve got something hardcoded that shouldn’t be.   

Create a free website or blog at

%d bloggers like this: