Before AI, search engines like Google would crawl websites. This means they would read your site’s info to show it in search results.
But with the coming of AI, companies started using special programs called AI crawlers. These programs automatically collect huge amounts of information from websites: text, articles, pictures, and more.
But here’s the problem:
- AI bots take content and show it in search results, but they don’t link back to the original website. This means the people who created the content don’t get visitors, don’t earn money, and don’t even get credit for their hard work.
- Content creators lose motivation. If no one sees or pays for original content, why would anyone want to create it?
Cloudflare’s new rules for AI
Cloudflare has begun blocking AI crawlers from looking at website content. Now, they can only do so if the site owner consents or if the AI company pays for it.
Not using Cloudflare? Here’s why you might want to
Cloudflare makes websites faster, safer, and easier for everyone to use. Let’s say your server is in Germany, and someone trying to visit your site is in Australia. The data has to travel a long way (and who knows better than you about long trips?), which makes your site slow. And if hackers anywhere in the world decide your site looks juicy, you might get attacked. Enter Cloudflare.
- Faster loading: Cloudflare has servers all over the world. When someone visits your website, Cloudflare sends them the information from the server closest to them. This makes your site load much more quickly.
- Stronger security: Cloudflare stops bad traffic, hacker attacks, and spam before they even get to your site.
Always available: If your site ever goes down, Cloudflare can show visitors a saved (cached) version. This means people can still see your content even if your site isn’t working.
How to regain control of your content with one click
Cloudflare now asks site owners their preferences with regard to AI bots:
This means:
- If you own a website, you get to decide if AI companies can use your content. You can even tell them how they can use it – like for teaching their AI, for search, or to create AI overview answers. (Now AI crawlers must declare why they want to collect data.)
- If you start a new website with Cloudflare, AI crawlers will be blocked by default.
- AI companies might start paying to use website content. This could mean that people who create content will finally get paid fairly for what they make!
How Cloudflare’s changes can help travel bloggers
Simply put, Cloudflare is giving website owners control over how AI uses their content. If you’re a blogger, this can help you in several ways.
1. Visitors will have a reason to keep visiting your site
Cloudflare’s changes could mean that people will need to visit your website more often to get the full, original information.
For example, let’s say someone asks an AI chatbot “What are the best hiking trails in the Dolomites?” If the AI system can’t easily scan your blog about the Dolomites, it’ll have to either link to your blog as a trusted source (if it makes a deal with Cloudflare and you) or give a lower-quality answer.
2. Your original content will be protected
Everything you’ve spent time and care creating – like a unique itinerary for Sicily or your original photos of a Bali sunset – may have been “eaten” by AI bots for free.
Cloudflare’s changes mean your content won’t be automatically scanned by AI bots without your permission. And it won’t be twisted into incorrect information in search results.
For example: If you’ve written a guide called “Iceland Road Trip: 7-Day Itinerary” with unique photos and tips, AI services won’t be able to simply take your whole itinerary and photos to train their systems and create answers for users without mentioning you. This helps protect the value and uniqueness of your hard work.
3. You can get paid if AI does use your content
Before, you didn’t get anything when AI services used your content – you just lost visitors and earnings.
Cloudflare’s Pay Per Crawl feature will make AI companies that need your content pay you to access it.
For example: You have a blog about food tours. You could sell access to your content for AI training. This means you would get money when these services scan your content.
Will I get penalized for blocking AI bots? Fortunately, it seems not. Travel bloggers who have already blocked AI bots haven’t reported any negative consequences.
Here’s what Marek Bronstring from Indie Traveler had to say:
“I’ve had AI bots blocked using Cloudflare for a long time. No negative impacts and I continue to be indexed and even listed as a source in AI answers.”
Has Cloudlfare just ended the age of AI thievery? Share your thoughts in the comments!
