Inside magnetic materials, there are regions called domains.

In un-magnetised materials, these domains are not lined up hence they cannot be a magnet.

Here is a diagram of domains within an un-magnetised piece of iron:

When you magnetise a piece of iron and turn it into a magnet, you cause the domains to line up.

Here is the diagram above, but now also showing the domains lined up when it has been magnetised to become a magnet.