Monday, May 24, 2021

The Amnesia Test

The Redo Test asks how much it would cost to redo a task, given that everything except the worker's memory of doing the task is lost. The Amnesia Test uses the opposite scenario: what if the worker forgets everything they learned from doing the task, but everything else is unaffected? How much would it cost to recover the worker's memory, given that the other results of the work remains? The Amnesia Cost Ratio (ACR) measures this as a percentage of the cost of doing the task the first time. An ACR of 100% means that nothing is useful without the worker's mental state, an ACR of 0% means that the worker can walk away (or be kicked out) without any loss of value, for the given task. 

Some examples:

1) You are a junior engineer working for a company that produces doohickeys. In evaluating a new prototype doohickey, a senior colleague asks you to check whether the prototype manifests "overextended flimming". You start writing a script to provoke overextended flimming, but get interrupted by a more pressing task. When you get time to work on the flimming again, you look at your script and draw a complete blank as to what you were trying to do. It is faster to just rewrite the script from scratch, rather than to salvage your old half-finished idea. The ACR for this (partially completed) task is 100%. 

2) You are the best welder in the doohickey factory, with 15 years experience. Every year, you have become 10% faster, which means that you can now weld a doohickey about 5 times faster than a new recruit. One day, you are hit by a tram, causing you to forget everything you knew about welding, so you have to start again. Let's assume that wages for doohickey-welders are always the same, and that learning saturates after 15 years. How many more doohickeys would you have welded in the following 15 years if you hadn't been hit by the tram? About twice as many. The ACR for this task is 50%. 

Number of iterations until saturation Improvement per iteration ACR
15 10% 50%
1 30% 30%
2 30% 40%
5 30% 61%
100 2.25% 61%
10 2.25% 12%

Difference between KCR and ACR

One important difference between the KCR and the ACR is how the incentives of employer and employee are aligned. The employer wants a high KCR, since it means that the capital is safe against loss of physical inventory, or disk crashes. The employees also want a high KCR, since it means that they themselves have increased their human capital; they could now do similar tasks faster. There is however a conflict between the employers and the employees when it comes to ACR. The employees want a high ACR, since it gives them leverage; they could leave and remove value from the company. The ideal situation for an employer would be a company whose main capital is an idea or a culture. Once understood, the idea or the culture can be used to create value for the company, but an employee cannot remove the idea or the culture by quitting. The issue with having an idea or a culture as the main capital is that a competitor might try to copy you. 

Consequences of the Redo Test

A task with a low Knowledge Cost Ratio (KCR) is by definition less robust to events that destroy all the results. The KCR decreases as the task is done more times, which decreases the robustness to unexpected loss. This gives a rationale for knowledge worker's reluctance for repetitive tasks. Since the KCR decreases with the task being done more times, we should expect that different people do the task later on. In the extreme case, the task gets automated. 

Some activities such as advertising and quality control are expensive and do not produce a physical product, which makes them superficially similar to knowledge work. However, they are not knowledge work according to the redo test.

Another consequence happens if you introduce procedure that requires artifacts from knowledge work to be saved, or else the work counts for naught. In that case, a lot of work may have to be redone unnecessarily.

The Redo Test

What is knowledge work? The Redo Test provides one answer, using a thought experiment. 

Suppose you have just completed some work. Now suppose that every result of this work disappeared, except your memory of it (including episodic, procedural memory, etc). How much would it now cost to redo the work a second time, as a percentage of what it cost to do the first time? Let's call this the Redo Cost Ratio (RCR). Let's also the define the Knowledge Cost Ratio (KCR) as 100% minus the RCR. A KCR of 0% means that there is no knowledge element to the result of the work, a KCR of 100% means that the only valuable result of the work is knowledge. 

Some examples:

1) You make a soup. You have made this soup a hundred times before, so you always do it in exactly 90 minutes. When you are on your way to serve the soup, you stumble and the soup spills all over the floor. Redoing the soup would take 90 minutes, so the RCR is 100%, and therefore the KCR is 0%. 

2) You find a bug in your software. You debug for 20 minutes, and finally find the cause. You implement a fix, which takes 5 minutes, but the source code for the fix is lost when you forget to save. Luckily, you still remember the cause of the bug, so you only have to rewrite the 5 minute fix. The KCR is 80%. 

3) You publish your first app to the App Store. You try 10 different advertising tactics, before one succeeds, and the app finally takes off. All of a sudden, your app is banned from the App Store due to an algorithmic mistake, with no ability to appeal. You have to republish the app under a new name and logo. The second time, it only takes a combination of 3 advertising strategies before the app takes off. The KCR is 70%. 

4) You represent a large company that releases one of your products in a new country. You spend $1 billion setting up: recruiting marketing and salespeople, renovating office space, etc. They then spend $1 billion per year for 9 years building and maintaining your brand using TV and magazine ads. Suddenly, an unfounded rumor causes your brand to become permanently socially undesirable in that particular country. You still have the people and the real estate, but you have to spend another $9 billion over 9 years to build the brand of one of your other products. The KCR is 10%. You have not performed knowledge work, but signalling work. Edit: depending on how you define it, KCR can also be 0. Every result of the work should be erased except the memory, and that includes the hires and real estate. 

5) You inspect a toy for safety hazards. The toy passes all your tests, but you lose the quality assurance documentation files on the computer. Your knowledge that the toy passed the tests are worth nothing if you couldn't present the files to the meta-inspector. The KCR is 0%. You have not performed knowledge work, but bureaucratic work. 

6) You are a junior engineer working for a company that produces doohickeys. In evaluating a new prototype doohickey, a senior colleague asks you to check whether the prototype manifests "overextended flimming". You take the prototype out for a spin, but forget to press 'record session'. The flimming is perfectly normal, however, and you tell the senior engineer as much. When you admit that you forgot to record the session, your colleague answers "That's OK, at least we know that we don't have to prioritize the flimming". The KCR is 100%.