Incident investigation
Finding a 9 GB check-in across a distributed clinic network.
A distributed dental group lost access to a web-based clinical system during the same three windows every day. More than 60 clinics shared the network, so one source could disrupt work everywhere while appearing to implicate the wrong platform.
- Environment
- 60+ clinics
- Managed estate
- 1,000+ endpoints
- Repeated transfer
- 9 GB per check-in
01
The visible failure pointed in the wrong direction.
The traffic appeared to implicate the clinical platform because both paths terminated in AWS. That correlation explained what users saw, but it did not identify the source of the saturation.
Full packet capture was not viable at this scale. The firewall appliance exhausted its available memory within seconds, so the investigation needed a different form of observability.
02
Build the evidence the incident requires.
I created a separate Azure log pipeline and compared destination traffic with private-link telemetry. That narrowed the activity to one network scope rather than one application.
DNS logging then exposed the relevant destination pattern. I used Wireshark at the endpoint to verify the behavior and connect the network symptom to its actual source.
- Separate Azure telemetry pipeline
- Destination and private-link traffic comparison
- DNS logging to narrow the source
- Endpoint verification with Wireshark
03
The root cause was a repeated management task.
A remote-management task downloaded the same 9 GB package at every check-in. Removing the affected user tasks ended the saturation across the network.
I reported the behavior to the vendor, and it was changed in a subsequent product update. The case remains anonymous; the useful evidence is the investigative method, not the client or vendor names.
When the existing tools cannot answer the question, the next step is often to design the observability the investigation needs.