Baryon

A baryon is a heavy subatomic particle, a hadron (a group of elementary particles that experience strong nuclear force) with a non-integer spin. They include protons and neutrons

Baryons accumulate aboard starships as a result of warp travel. They can be removed at facilities that perform baryon sweeps, a procedure lethal to organic material.

As almost all normal matter in the universe contains these particles, a process which eliminates baryons wouldn't leave much behind. We can assume that the baryon sweep refers to a specific type of heavy subatomic particle created by the subspace field of warp drive, rather than the entire family of them. A baryon sweep would thus indeed be dangerous to organic matter, for the reason stated above.

Fun Fact:

When Quinn transported the USS Voyager to just before the "Big Bang", a build-up of baryonic particles was detected outside.