# Using Netlify I use Netlify to build and host my static websites, built using Hugo or MkDocs. They have a generous free tier, and provides a feature that I find invaluable: *the ability to preview pull requests*. This is one feature that stops me from using GitHub Pages. ## Blocking AI crawlers While we can add `robots.txt` from this [Github repo which tracks the updated list of AI crawlers](https://github.com/ai-robots-txt/ai.robots.txt), `robots.txt` is advisory and not great for handling crawlers that don't follow the rules. Netlify provides a feature called edge functions, which allows you to run JavaScript code at the edge. This is useful for customizing the behavior of your site, such as blocking certain crawlers based on their user agent. This is marginally better than `robots.txt`, as it relies on the user agent HTTP header, which is more widely observed by legit crawlers. i.e. it relies on identity and not behavior. [Jeremia has a great article on how to do this.]( https://www.jeremiak.com/blog/block-bots-netlify-edge-functions/) ## Gotchas ### Lowercased file names I found this out when I was building this site and I wanted to use camelcase file names. Even though locally the file names were camelcase, on the built site they were all lowercased. This is apparently [an undocumented behavior of Netlify][1]. Credit: [jvt.me](https://www.jvt.me/posts/2019/11/11/gotcha-netlify-lowercase/) [1]: https://community.netlify.com/t/my-url-paths-are-forced-into-lowercase/1659/2