Suppose you learned a decent amount of French at school. Open almost any language app and it will still start you at hello. You can order coffee in Lyon, but the first lesson is greetings, because the app does not know what you can do and does not try to find out.
The usual fix is a level test with a result at the end. We built one of those too and threw it away. "Upper intermediate" is something you can say at a party, but it does not tell you what to do on Tuesday. We replaced the test with a check, and this post explains what the check does.
It looks for the first thing you cannot confirm
The check is not an exam on the whole subject. It is a search for a boundary: the point where what you can do ends.
It starts with a question from the middle of the route. If you answer it, everything that clearly comes before it is taken as confirmed too, and the check moves forward. If you do not, the check moves back. After a handful of questions it has found a place: the first thing on the route that you cannot do yet. Your first lesson starts there.
This is why the check is short and why it feels uneven. One question will seem far too easy and the next one too hard. That is how a search looks to the person being asked. The check is not trying to cover the material evenly. It is trying to find one boundary and then stop asking.
Hints are switched off, and only here
Everywhere else in the product you can ask for help. Hints come one at a time, the mentor explains the part you got wrong, and asking for help does not count as failing.
In the check none of that is available. It is the only place where EvoluteMe is deliberately unfriendly.
A hint changes the question. "Can you do this" becomes "can you do this with a nudge", and for almost everybody the answer to the second question is yes on almost any topic. The check would then put your starting point further along than you really are. The first lesson would rely on things you do not know, and every step of the route after it would carry the same error.
The two kinds of error do not cost the same. If you start too early, you are bored for an evening. If you start too far ahead, you may decide that you are bad at the subject, when in fact a program guessed wrong about you in the first two minutes.
Nothing is scored and nothing turns red
There is no mark at the end: no score, no band, no certificate. You get a sentence saying where you are and a route with a point marked on it.
Wrong answers in the check are not counted against you anywhere. A wrong answer changes the direction of the search and nothing else. No screen adds them up, no parent sees a tally, and no team lead reads them. The same goes for a hint you take in a lesson: it stays in that lesson and is shown nowhere else.
We are not being squeamish. Once a check produces a mark, people treat it as a mark, and the sensible response to being marked is to be careful. A careful answer is really a guess, and a guess breaks the search. The check only works while it is truly safe to say that you do not know.
What a confirmed skill gives you
A confirmed skill is behind you. It never comes back as a lesson, and you will not spend an evening proving to a product that you can still do something you could do on Monday.
It can come back in one form only: a short review, when a later step relies on the skill and your recent work suggests it has become shaky. This is not a scheduled revision, a weekly quiz, or a reminder that you have not practised for a while. The review happens only because the next step needs the skill.
This also answers the question people ask most often about the check: what happens if they guessed well and were placed too far ahead? The route finds out almost immediately, because the next step relies on the skill. The walkthrough goes to the point where your reasoning went wrong, and you get a short review of the missing piece. You are not moved back down the route.
What it cannot see
Anything that cannot be answered at a keyboard or out loud.
Whether you can really hold a conversation in a language with somebody who interrupts you. The check can hear you say a sentence and tell whether you managed to say it. It cannot put you in a room with a stranger, and we do not pretend that speaking to the mentor tests the same thing.
Whether you can do something with your hands. A hairdresser, a car mechanic and a nurse each have skills that no question on a screen can test, and those skills are a large part of the job. The photographed drill helps a little, because a photograph of your working is real evidence of how you reached the answer. But it shows your working, not your hands.
Whether you can do the thing in the conditions that make it hard: tired, in a hurry, with a client waiting. That is the gap between a confirmed skill and a job. It is why a route ends when you can do the thing, and not with a promise of employment.
Where we might be wrong
The check is short because people give up on a long one, and a short check is not precise. On a big subject it can put your starting point a little earlier than it should be, which costs you a couple of easy evenings. We chose that error on purpose over the opposite one.
The more serious criticism is that a boundary simplifies a person. Knowledge is not a line with everything before it solid and everything after it missing. Real people have gaps before the boundary, left years ago by a teacher who was ill for a fortnight.
The review of shaky parts is how we catch those gaps, and it catches them late: only when a later step relies on the missing piece. We have not found a way to look for them directly without ending up with the long test we refused to build.
