I was avoiding doing the collision detection in Unity at first because I thought it might be quite involved with loads of Vector Math... When I got around to doing it I was, one: very surprised at how easy it was and, two: extremely impressed with the implementation of collisions in Unity.
This is what I did:
1) I created a physic material, called it Rock (The name isn't important) and had a play about with the settings. My settings were: 0.8 for Dynamic Friction, Static Friction and Bouncyness. Average for Friction Combine and Bounce Combine.
2) In my asteroid prefab I added a capsule collider and set the Material to the Rock physic material I just created. I had to set the capsule collider radius, height and direction to match the shape and size of my asteroid mesh.
3) Sat amazed at how easy and effective this was in Unity!!!