I wrote the first spec for the ImageSim rebuild in the fall of 2019. Version 1.6, finished the week before Christmas, prepared for Dr. Martin Pecaric and Dr. Kathy Boutis. The job on paper was straightforward enough: their medical imaging education platform ran on Flash, Flash was about to die, and they needed a modern web application that could carry years of existing content across without losing anything.

We’re still working on it seven years later, and it’s turned into one of the more interesting things we’ve built. Not because the technology is exotic, but because the requirements came from somewhere most software requirements don’t come from.

What ImageSim actually is

If you go to an emergency department with a suspected fracture, somebody reads the X-ray. Usually that somebody isn’t a radiologist. It’s an emergency physician, a resident, a family doctor. They make the call and you get treated based on it.

The gap between what a front-line clinician sees in an image and what a specialist sees is a known source of medical error. It’s worse in kids, because children’s bones change as they grow, and a growth plate on a seven-year-old can look an awful lot like a fracture if you haven’t seen a few hundred of them.

The normal way to close that gap is experience. Years of it, one patient at a time, with feedback that shows up late or never. ImageSim compresses that. You sit down and work through hundreds of real cases in a row. Every case, you have to commit: normal or abnormal. If abnormal, point to it on the image. You find out immediately whether you were right, and you see what the radiologist said. Then you do it again.

Kathy Boutis and her colleagues at SickKids and the University of Toronto spent roughly eleven years researching whether that works before any of it became a product. They published on how many cases it takes before performance plateaus, on what ratio of normal to abnormal images teaches best, on whether the order cases appear affects how people read them (it does), on what kind of feedback helps and what kind doesn’t. This year, the Canadian Association of Emergency Physicians named her its 2026 Ian Stiell Researcher of the Year.

So by the time we got the brief, the pedagogy was settled. Our job was to not break it.

Competency scoring: accuracy, sensitivity and specificity

Most learning platforms score you one way. Right answers over total answers, done.

The ImageSim spec has three numbers. Accuracy is what you’d expect. Sensitivity is how well you catch the abnormal cases. Specificity is how well you leave the normal ones alone. To be marked competent in a course, you have to clear the threshold on all three at the same time.

It took me a while to appreciate how much is packed into that. If you call everything abnormal, your sensitivity is perfect and you’re a menace. Every kid gets a CT scan they don’t need. If you call everything normal, your specificity is perfect and you’re missing fractures. Only all three together describe someone you’d want reading your child’s X-ray.

There’s a second rule in the spec that I still think about. Once you hit competency, it sticks. If you keep practicing afterwards and your running average dips back below the line, you stay competent. Marty was clear about that, and he was right. Build it the other way and you’ve made a system that punishes people for practicing past the point where they’ve proven themselves, which is exactly the opposite of what the whole thing is for.

Later I found the published study that the benchmark came from. A multicentre trial across nine emergency medicine programs in the US and Canada, 80% on accuracy, sensitivity and specificity, measuring how many cases fellows needed to get there. The rule in our database is the rule in the paper. That’s not a coincidence, and I don’t think I fully understood at the time that it wasn’t.

Engineering challenges in an assessment platform

The answer to the key problem was in the spec from day one, and it’s the thing that makes an assessment platform different from a course platform. If the case data sitting in the browser includes the correct diagnosis, anyone with dev tools open can score 100%. That doesn’t just cheapen the certificate, it poisons the research, because the whole system is also a data collection instrument and a contaminated dataset is worse than no dataset.

Marking up where the abnormality is turned out to be a real piece of engineering too. Asking “is this normal” is a yes/no. Asking “where” means capturing a coordinate on an image the learner may have zoomed, panned and resized, on anything from a phone on a night shift to a proper workstation, and then scoring it against a region a radiologist drew months earlier. We built the authoring side of that on Fabric.js so clinicians could draw hotspots and annotations straight onto the image in a browser instead of shipping us files.

The case loader is where I got a useful correction. I’d specced a full front-end course builder. Dr. Pecaric’s comment on the draft was that setting up an entire course through a UI was going to be more complicated than it looked, since you have to handle which cases belong to a course, adding and removing them, how modules relate, and that what they actually needed urgently was a good way to create and mark up cases. He suggested doing course assembly in another way and revisiting. That was the right call, and we went with it.

Then there’s the unglamorous half of the spec, which is the half that bites you. Surnames longer than thirty characters. Database collation that handles French and Spanish keyboard input. Email uniqueness and case-insensitive lookup. A bandwidth floor of about 1 Mbps and a battery budget, six hours on a phone under constant use. At the time those read like boilerplate to me. They’re not. The platform is used in more than 65 countries, including places where the connection is bad and the nearest radiologist is very far away, and it’s delivering high-resolution diagnostic images and now ultrasound video. Those numbers were somebody thinking ahead.

We also built out eight user types, which sounds like over-engineering until you see how medical education actually buys training. A program director purchases a block of seats for their cohort and assigns them by email. A coordinator handles the logistics but shouldn’t see anyone’s grades. A manager runs the day-to-day without spending money. Content authorship gets delegated per course to whichever clinician owns that subject. All of that had to coexist with a single physician buying one course with a credit card on a Sunday night.

Where it is now

The Flash app had a handful of radiograph courses. The catalogue today is past thirty.

Chest radiographs for adults and kids. Seven musculoskeletal regions, most with a standard and an advanced version where the advanced one makes you risk-stratify the fracture and decide on management, not just spot it. ECG. Point-of-care ultrasound including cardiac, ocular and first-trimester obstetric. Pediatric dermatology. GI polyps. Dental bitewings. Identifying burns and bruises that suggest child abuse. Courses run from about 100 cases to over 400, and the competency threshold varies by course depending on what it costs to be wrong: 80% on bitewings, 85% on chest films and ECG, 90% on the elbow, the pelvis, the ultrasound courses and the abuse identification ones.

It’s accredited by the Royal College, the College of Family Physicians of Canada, and the American Board of Pediatrics, with reciprocal arrangements in Europe and Qatar. Somewhere north of 3,000 clinicians and 40 postgraduate programs use it. Course sales subsidize access for clinicians in places that can’t pay, which is the sort of thing that makes a project easy to say yes to.

And it’s still producing research. A study published in Academic Radiology this year pulled 48,166 case interpretations out of the platform from 100 radiology trainees. Between their first 25 cases and their last 25, sensitivity went up about 15%, localization accuracy about 14%, and risk stratification nearly 24%. Three-quarters of them hit the standard in at least one body region, taking a median of 173 cases, which works out to around 41 minutes of practice. The same data showed where people go wrong most: elbows, pelvises and ankles, and everything gets harder when abuse is suspected.

That last part is the bit I’d point to if someone asked why this project is different. Those aren’t product metrics. They’re findings about how doctors learn, and they came out of software we built and help maintain. That means every change we make has two audiences. A schema change for a new course can’t quietly break a dataset somebody is three years into collecting. A performance tweak can’t alter the conditions a published result depended on.

Seven years in, the useful thing we bring isn’t the original build. It’s that we know the history well enough to change a live system without invalidating what’s already been said about it.

We’ve been building healthcare software since 2013 — ISO 9001 and ISO 27001 certified, Ontario Ministry of Health certified integrator. If you’re working on something similar, just reach out!