I thought I would give a few suggestions.
Babies and toddlers
Percussion instruments are the most suitable and can be enjoyed from almost birth. This page contains links to real instruments, and not toys, so very young children would need supervising.
These are simply 2 short, thick sticks that you hit together. You can make your own out of a broom handle, but the real thing will make a much better sound.
Played with a wooden beater, very easy to play.
This is quite versatile. It can be scraped and hit with the wooden beater. It gives 2 different tones.
This single guiro style tone block is very sturdy, and the beater is made from a single piece of wood, so no round balls stuck on- nice and safe for even very young babies.
A favourite! Hold by the cord and stike with the metel beater.
If you decide to get a castinet, the ones with handles are far easier to play.
Very easy to play. Older children can make some from yoghurt pots and dried peas!
Again, very easy to play. Can be played with beaters, or with your hand. I sometimes let very young ones play using a 
maraca as a beater. At the time of writing this, these bongos were in a sale and at very good price.
Can be shaken or tapped.
A tuned persussion instruments. They can be separated, so you only need to give 1 note to a child. I have used these with very young children.
Sometimes percussion sets are good value. These 2 have a good selection.
Key Stage 1 (ages 5 +)
All of the above instruments can also be enjoyed, but you could also try…
I wouldn’t get a toy one. I was bought one from ELC that cost £10. It was pretty out of tune. These ones are cheaper, and make a far better sound. Recorders come in different sizes and keys. Usually children start with “descant recorders”, which are in C. Children of 5 will find a recorder a challange, but should manage a few notes (B, A, G). It’s surprising how much you can play with just these notes.
I have used this book before (with KS2 children) and I thought it was good fun. The
first track was a poem that needs accompanying with sound effects- just blowing down the recorder with no fingers- an easy start and children love it! I have also heard very good things about the John Pitts books.
Books with CD accompaniments are great and can make even a beginner sound great!
Key Stage 2 (ages 7+)
Recorders are still more than suitable and make an excellent starter to many woodwind instruments.
These are becoming very popular, and are considered to be the “new recorder” in many primary schools. They are easy to play and cheap to buy. My youngest, aged 4, has a Uke, can play a C chord, and enjoys strumming along to nursery rhymes. They are great for adults who want to share music with their children too. You could learn a few chords and be strumming away within a couple of hours practice- honest. There are many good sites as well that give free tuition.
Ukuleles come in different sizes and keys. The most common ukes are soparnos. They are great for young children and adults.
You can buy an instrument from £15 or so, but Makala Ukuleles are very good starter instruments with pretty good tuning. My son has this one in light blue- costs just over £25. It has a good sound and is robust – not sure what it’s made from, but has with a rosewood bridge and finger board. Would definitely recommend. I have a slightly more expensive one, made from wood, but actually I don’t think the sound is as good.
Also, at the time of writing this, these ukuleles were on special offer- they come with 2 free books, a very good buy.
I have this book. I like it. It has tuning notes on the CD and very clear instructions. It has some basic musical theory, but is easy to follow. The songs are also well known, so you can sing along. Like many books though, there aren’t many tracks with just one chord (in this case, only 1 track). Fine if it’s for a KS2 child, but I found that I needed more examples for myittle one. I have recorded a few more single chord songs on the piano . If you can master C, G7 and F, you can play most of this book.