It is important to create and optimize the robots.txt to make your Magento store secure and improve SEO.
The robots.txt ("robots dot text") is a text file that help Search engine robots (such as Google bot and Bing bot) to determine which information to index. By default there is no robots.txt in Magento Community or Enterprise distributive so you should create it yourself.
See more