By David Hains
With the coronavirus pandemic growing rapidly around the world, jurisdictions are paying close attention to daily data to understand just how fast it's spreading, where it's coming from, and whether it's on a trajectory to trigger significant crises in the health care system.
Here we will walk through where readers can see this daily data and help to explain what the numbers mean and the additional questions that can be derived from them.
That said, there are limits to this information. It is not always easy to compare between countries or even provinces, as test availability and parameters can differ significantly. Who gets tested has evolved within Ontario since the first positive case in late January as Public Health Ontario's testing capacity failed to keep up with the spread of the virus. And because of the period of a number of days before people with COVID-19 show symptoms, and the backlog of tests, the daily numbers are best seen as a lagging indicator of where things were at several days ago.
With those caveats in mind, here's a breakdown of how to interpret the data and understand the limits of what it's telling or is not telling us.
Where to find the data and what it's measuring
Every day the Ministry of Health posts an update around 10:30 a.m. that outlines the number of cases, including how many are currently positive, resolved and deceased, with data provided by the daily epidemiological summary analysis provided by Public Health Ontario. They also provide information on the total number of people tested to date and and how many are currently under investigation — which is the number of cases with test results pending, otherwise known as the backlog.
This daily update mostly forms the basis of much of the reporting on the rate of how the virus is spreading, whether sufficient testing is being conducted, and, of course, on the fatalities.
Like any daily update, the data is a snapshot in time. One shortcoming of the data is due to two delays: the amount of time that it takes for the test results to process, and the amount of time it takes for symptoms to show in patients, and thus qualify for testing.
As of last week the daily data showed that there was a backlog of tests that was greater than 10,000, with fewer than 3,000 tests being done per day. That created an issue in that the information being provided to both individual patients and the public wasn't particularly timely.
This week Public Health Ontario has turned the corner on the number of daily tests, including performing over 6,000 daily tests in recent days, in part because of additional labs and assessment centres being brought on board to add capacity. This has greatly alleviated the backlog over the past few days, which now stands at 2,052.
Another aspect that makes the summary data a lagging indicator is the nature of the virus. Because it takes a few days following transmission before symptoms materialize — it can be around five days, according to a study in the Annals of Internal Medicine — the daily summary results should be understood as a backwards-looking exercise.
There's also a small reporting delay too. The daily numbers are gathered by public health units sending their data to the reportable disease database and compiled by the Ministry of Health. Those numbers represent 4 p.m. the previous day, and they are then gathered and released around 10:30 a.m. the following morning.
Between the delays in testing, how long it takes for symptoms to arise and the time it takes to gather reporting the numbers should not be interpreted as the number of cases in the public on that day, but a number that more closely reflects where things were at several days ago instead.
The limits to the summary data
One limit to the data is that not everyone who has COVID-19 gets tested for the virus.
This, says critical care physician Dr. Kali Barrett, means that we have an incomplete picture of how widespread the virus is.
"The number of cases we identify is a reflection of testing, not transmission," the Toronto Western Hospital doctor told QP Briefing. She points out that in China about 80 per cent of people who tested positive had only mild symptoms, while 20 per cent were severe. But the limits to testing capacity prevents Ontario from taking a more widespread approach at the moment.
Those decisions on who gets tested are made on a case-by-case basis and have been influenced by a shortage of testing swabs, among other reasons.
Today, the vast majority of the testing is done at hospitals, at 86 assessment centres set up by hospitals around the province, and at long-term care homes where outbreaks are suspected.
Dr. Raj Waghmare, who has been doing shifts at the COVID-19 assessment centre established by his hospital in the Toronto area, told QP Briefing two weeks ago that he and his colleagues were having to make tough decisions on the fly about who to test for the novel coronavirus because of a worldwide shortage of the swabs, and no clear guidance from Public Health Ontario.
At the time, doctors weren’t able to test everyone who was referred to the centre with symptoms consistent with COVID-19, because they would run out of swabs. So, each hospital, or each individual doctor, was creating criteria for who should be swabbed, such as those who are high risk because they are older and have other health problems, and health-care workers.
But on the whole, the swab shortage meant people who had symptoms consistent with COVID-19 and no other risk factor weren’t getting tested, and were simply told to go home and self-quarantine.
This week, Waghmare said the swab shortage has been mostly alleviated so he and his colleagues are able to test more broadly. One doctor at his hospital has been monitoring the swab supply that is available for use for patients who are admitted, as well people who are referred to the assessment centre, and keeping the doctors making those decisions informed so they can decide who to swab.
"It’s changing on a day-to-day basis," he said.
In general, people who come to the assessment centre who have no symptoms aren’t swabbed — some people will asked to be tested if they have no symptoms and no real connection to a positive case, he said.
Waghmare said it isn’t an absolute rule, but in general people who have COVID-19 symptoms, and not just those in those in high-risk groups, are now being swabbed and the supply of swabs should allow that to continue.
A corollary to the testing shortages and limits is that it becomes difficult to compare results across jurisdictions for several reasons.
Testing in one jurisdiction may be significantly more strict than in another, so it may look like they have fewer total infections, but a higher rate of positive tests because they aren't testing as many people.
For this reason, Barrett suggests that a more critical number to analyze is the number of people infected who are in intensive care unit beds, and that this can be used as a proxy to estimate the infection rate as a whole. "That is the number to track," she argues, an approach that's backed up by Alfred Whitehead, the executive vice-president of applied sciences at Klick Health.
Dr. Shelley Deeks of Public Health Ontario agrees that the summary data across jurisdictions isn't always consistent because of the different ways of collecting and measuring the information. However, she says that the best measure to compare apples to apples is fatality information, as there can be difference in health care systems that can skew the ICU bed information too.
Even among fatalities it's not always an even comparison. The Toronto Star reported that the data shared by Public Health Ontario differs from the total reported by public health units, with the discrepancy equalling as much as 100 per cent. When asked about the difference Associate Chief Medical Officer of Health Dr. Barbara Yaffe said she was surprised, and that the province at this point doesn't have a better way to report the numbers.
Fatalities also don't take demographic information into account. While COVID-19 can infect and kill people of all ages, seniors are especially susceptible to the complications from the illness, according to a March paper in the peer-reviewed journal The Lancet. That's one of the reasons why Spain and Italy, two countries with aging populations and high fatality totals, have had such a difficult time with the coronavirus, notes Barrett.
One other limit in the summary data is that the criteria for testing has evolved over time, so it's not a perfect comparison even over the 50-plus days of data that Ontario currently has, according to one health expert who spoke on background.
That said, there are some useful parts to the summary data. Barrett says that the day-over-day increase in the numbers can help illustrate a story as to the epidemiological curve, and that people should be paying attention to the rate of the increase. Ontario has thus far avoided daily increases in positive cases like the 33 per cent seen in the Italy scenario, and has generally fallen between 15 and 25 per cent increases per day.
Going beyond the summary data
The province is now sharing more information about the cases that make up the summary data. It is making a daily open data set available, and is providing details on the ages of the people who have tested positive. There is also an epidemiological report covering information from Jan. 15 to Mar. 31, including where in the province people are testing positive, how many have been traveling from abroad, where they traveled from and more.
Some of this information can be critical, as it can illustrate what percentage of transmission that we know of so far is due to community spread as opposed to known influences like travel — the higher the rate of community spread the worse it is for a given jurisdiction.
As of Thursday morning, the daily data release includes the total number of ICU beds and ventilators being used by coronavirus patients in Ontario. Over the past week or so this information had been shared orally by Dr. David Williams in the daily press briefings that occur at 3 p.m.
It's also information that key government decision-makers at the command table have had at their disposal, as Critical Care Services Ontario, a branch of the Ministry of Health, provides a detailed breakdown of the number of ICU beds and ventilators that are occupied and available in different regions.
Internal numbers were recently shared with the CBC to highlight the current condition of hospitals and their needs, and some screenshots of the data have been leaked to Twitter.
Critical Care #Ontario Mar 30📈:
2012 crit care beds w/ 1559 crit care pts (77.5% capacity)
125 confirmed #COVID19 (8.0%)
353 suspected #Covid_19 (22.6%)241 confirmed/suspected #Covid_19 w/ invasive ventilation
This is just the beginning - please #StayHome & #FlattenTheCurve! pic.twitter.com/LksTtEdj2g
— Dr. Jennifer Kwan (@jkwan_md) April 1, 2020
At the same time, a group of health experts, academics and data scientists are working on the site HowsMyFlattening.ca to consistently compile open data and visualizations to portray Ontario's current health system condition and capacity and help inform decision-makers. The available data includes numbers on Ontario, Canada and worldwide cases as well as information on ICU beds and ventilators within the province, pulled from a PDF and converted into a manageable spreadsheet form.
A visualization produced by a team at HowsMyFlattening.ca. The team warns that a high proportion of people who have coronavirus are not identified as such, based on the experience in Wuhan. And epidemiologist David Fisman cautions on Twitter not to necessarily interpret this data as a flattening of the curve, because recent cases take several days to be diagnosed.
Whitehead, a data scientist, is part of this group of over 70 people, and he makes the case for why it's important to be fast and granular with the data. "A pandemic is a million local emergencies that happen everywhere," he told QP Briefing. That could mean that it's more acute in a one Ontario region than another, and health care system capacity may have to be nimble to recognize and respond to those needs.
For instance, one city or region may have adequate ventilator capacity while another is short — and if Kingston has sufficient capacity that may not be of much solace to patients in, say, Kenora. In order to have better analysis and more minds working on the problem, he also advocated for "radical transparency" when it comes to the government releasing and sharing its numbers.
While this Ontario map of reported coronavirus cases from the epidemiological report highlights different areas of the province, it doesn't adjust based on per capita results or local health system capacity.
There's a word of caution with the ICU bed data, though. While the number of "suspected" COVID-19 cases is quite high compared to the number of confirmed cases, Barrett explains that the numbers play it on the safe side. The suspected cases include all patients who have a pending test or any respiratory illness. As well all patients are tested before they go into the operating room, further increasing the number of suspected cases. The good news, she adds, is that hospital tests tend to be fast and are generally completed within 24 hours.
Understanding the models
There are a lot of models and projections making the rounds that portray potential outcomes depending on certain actions like aggressive social distancing versus nothing at all. Like any models, these are based on any number of assumptions and should be used as a guide to inform decision-making and potential risks.
In the best scenarios the rate of increase slows before the peak is reached and the curve flattens. This can spread the amount of time the coronavirus is in a given environment, which sounds like a bad thing, but it means that the health care system is more able to respond to needs.
By contrast, in worst-case scenarios the curves are characterized by high daily increases leading to an early and sharp peak. This surge in cases can overwhelm the health care system, where there's only so many ICU beds, ventilators and trained medical staff to meet the needs. Exceeding that capacity would mean that many people who need ventilators (although the government is working on procuring more) would not get them.
Ontario's daily growth rate in positive cases has tended to hover between the 15 to 25 per cent range. The longer-term trend is what is significant here rather than any given day, as Williams has cautioned against reading too much into one day's data.
A team of health experts from the University Health Network and the University of Toronto, among other institutions, has provided some potential scenarios to understand Ontario's situation. That included initially modelling Italy's 33 per cent growth rate scenario and a much-better 7.5 per cent scenario. A couple days later the model was updated with 25 per cent and 15 per cent scenarios, which are closer to what the daily summaries — albeit data that has its limits — track the province's growth rate at.
Understanding the models is important to anticipate what the province's needs could be, and to address them before they become critical. In the 25 per cent scenario, which is slightly higher than the reported numbers Ontario has been seeing, the province would see depletion of ventilators and ICU beds in early-to-mid April. Accounting for the effects of social distancing, ICU beds would then be freed up, although there wouldn't be enough ventilators thereafter. In a more conservative scenario this resource depletion wouldn't occur until late-April, which highlights why experts and officials have repeatedly said why the next few weeks are so critical to fighting COVID-19. This modelling was also based on the number of ICU beds and ventilators as of mid-March, and the province has worked to add capacity since then.
On Wednesday, the chief medical officer of health said he intends to share some information derived from projections or modelling, but not until sometime next week. Williams said Ontario's data is still too uncertain to give reliable figures to the public, cautioning that small variations in the data that goes into a model at this point lead to wildly different results.
The premier and health minister were asked on Wednesday about what projections they had seen, but declined to share the numbers, although British Columbia has done so without detailing fatalities. This comes as the U.S. government shared that even with social distancing the country is looking at the range of 100,000 to 200,000 fatalities due to the coronavirus.
Although US officials have shared #COVID19 model showing 100,000-200,000 deaths, Ontario Health Minister @celliottability evades direct question from @ColinDMello on whether she's seen a projection for here. "A model is just that. A model." So, yes but it's a secret? #onpoli
— Rob Ferguson (@robferguson1) April 1, 2020
On Thursday afternoon the premier announced a pivot, saying that the government would release its modelling information to the public on Friday. He warned the public that the numbers would be "stark" and "sobering," but expressed his hope that transparently sharing the data would demonstrate to Ontarians the value of social distancing and public health interventions.
-With files from Jessica Smith Cross
Leave a Reply
You must be logged in to post a comment.