Health
Tap water is drinkable across the islands. Your chances of getting food poisoning are the same as anywhere in the UK, ie: pretty low. Orkney is fairly low-lying, so generally it is not advisable to drink water from streams as they may only be rarely above human habitation. That said, you can always gather the water and drop a chlorine tablet into your bottle, as you might elsewhere in the world.
Midges certainly flex their muscles in Orkney, though there is nothing remotely approaching the formidable squadrons found in the Highlands and on Skye, or even on the Outer Hebrides. The most vulnerable to the beasties are those who fish in lochs on a calm day. Of more concern should be horseflies, known locally as cleggs, whose bites can cause horrendous swellings. Be mindful also of sheep ticks, or kebs, should you rummage around in sheep wool at factories or farms. Your chances of contracting orf are low but your GP may be slightly excited and intrigued should you return with the disease, which manifests itself with a mild fever and scabby, puss-filled lesions.
One nasty disease that ticks are increasingly transmitting is Lyme, a bacterial infection often manifested by a circular rash and/or flu-like symptoms and muscle and joint pain. The NHS website provides good information on the disease.
Safety
Orkney is an extremely safe place to visit and you are unlikely to have any kind of negative experience on the islands. Crime is extremely low (though the theft of Stations of the Cross from the Italian Chapel in 2014 stunned everyone.
The circumstances in which you should be mindful of risk emerge when you explore the great outdoors. When walking, you should certainly be wary of cattle, which will not be as well versed in the rights of open access as you might be. Avoid getting between a calf and its mother and be particularly cautious in May when most cattle are finally let out of their barns after being cooped up for the best part of eight months. Every year there is a handful of incidents involving cattle trampling people (usually farmers). Use your judgement and certainly keep dogs on a lead when in any field with livestock. Rusty and sharp-edged farm gates are ubiquitous and an easy way to scratch yourself and introduce infection.
Only on Hoy, with its high hills and deep valleys, is the combination of remoteness and bad weather a safety consideration when walking. Always be mindful of cliffs, which can be remarkably adjacent to coastal footpaths in a way you rarely see anywhere in, say, Devon and Cornwall. Another landscape feature that Orkney possesses is the ominous-sounding gloup, a large hole near cliffs that drops to the sea. These are very rarely fenced off. Needless to say, avoid walking in fog in their proximity.
Always ask locally which beaches are safe for swimming. The same goes for quicksand, which is rare but can come and go depending on how much water is carried by a burn to the beach.