Meta can crawl business websites under its terms, but that does not prove it is training AI on your website. Meta website crawling terms need a close look. What the terms allow, what bots fetch and what goes into AI training are three different things.
For Malaysian and Singaporean SMEs, there is no need to panic. Check which terms apply, what your site exposes and which crawlers you want to allow. Write down choices with your marketing team and developer. This review uses wording available on 26 October 2026, before the planned November changes.
What do Meta website crawling terms change?
The November previews cover crawling in more cases. But they are not the first terms to let Meta crawl. Read the current terms and previews separately to check what changes for your business.
Separate today’s terms from November’s previews
The current Commercial Terms took effect on 6 March 2024. The current Self-Serve Ad Terms took effect on 22 August 2025. Under those ad terms, placing an ad order already lets Meta access, index, cache, analyse or crawl your domains, webpages or URLs. Listed uses include research and training, developing ads and business products, and product integrity.
If your business buys ads, do not assume the right to crawl starts with November’s update. The current general Commercial Terms and ad terms are separate documents, not the same rules.
Both the Commercial Terms preview and Self-Serve Ad Terms preview now show 20 November 2026. Reuters reported the extension on 23 October, which pushed back the earlier date of 30 October.
Read the retained wording, not an old screenshot
The previews still refer to crawling by bots and test login details. They cover linked domains and URLs made available through your use, or users’ use, of Meta products. They retain security and integrity purposes. The Commercial Terms preview covers business use broadly, including running a Page, not just buying ads.
These exact words describe one set of uses in the Commercial Terms preview, not the full grant of rights:
provide, develop, improve, research, and maintain advertising and business products
An earlier preview named training as a use. Nieman Lab documented its removal on 17 October. Taking out that sentence does not prove all possible AI uses tied to business have gone. Nor does it prove Meta trained a model on your own website.
Nieman Lab spoke to three legal experts who still had doubts about the change. These are views from a small group, not a survey that speaks for all experts or a legal ruling. Their concerns merit review, but do not prove how Meta used your pages.
Does crawling mean Meta is training AI on your site?
No. Crawling means software fetches content from a site. That alone does not tell you what Meta does with it next. Meta website crawling terms set out what Meta may do. Server logs show requests to your site. Neither gives you a full record of AI training.
Distinguish the crawlers before drawing conclusions
Meta’s crawler documentation, updated 21 May 2026, gives agents different roles. Meta-ExternalAgent covers uses such as foundation-model training or direct content indexing. Meta-ExternalAds supports ads and other business products. Meta-WebIndexer improves Meta AI search results and helps Meta cite and link to content.
FacebookExternalHit gets details for shared-link previews. Meta-ExternalFetcher fetches single links at a user’s request. Its roles include testing and improving agentic AI. Treating all Meta-labelled visits as the same task leads to weak claims.
Ask your developer to check requested URLs, times, response codes and any network evidence. A user-agent name is a clue, but software can copy it. Even a verified request cannot show every later use of the content.
Keep ownership questions separate
The right to crawl and who owns copyright are separate questions. Your site may mix staff copy, customer reviews, licensed photos and suppliers’ text. The right to post it does not settle every reuse question. Our guide to copyright in AI-generated art gives context, but does not decide Meta’s rights over your site.
Use clear words with your team: “We observed this crawler on these pages,” rather than “Meta trained on our entire site.” This supports sound choices without claiming more than the evidence shows.
What can robots.txt and website controls actually do?
Robots.txt tells crawlers your choices. Login checks and access controls protect private content. Each has a role. Do not treat a rule for bots as a lock on your site.
Choose preferences by purpose
Start with your content and business needs. A shop may want people to find its pages and see link previews, but want to limit foundation-model crawling. An advice firm may need tighter checks on customer files. Each Meta agent need not get the same rule.
Use Meta website crawling terms and developer guides to help choose. To restrict Meta-ExternalAgent, ask your developer to set rules for that user-agent group. Assess Meta-ExternalAds, Meta-WebIndexer and link-preview access separately. Test changes with your campaigns and how people share links.
Meta says crawler rules may take up to 24 hours to work. Its crawlers can store robots.txt for that long. This is a published limit, not a measured gain in privacy or ad results. Check logs after that time; do not expect an instant change.
Understand the exceptions and limits
Meta says Meta-ExternalFetcher may skip robots.txt for user-requested fetches. FacebookExternalHit may also skip it for security or integrity checks. A disallow rule is not grounds to promise that no Meta request will ever reach your site.
The Internet Engineering Task Force’s RFC 9309 makes this point about security:
These rules are not a form of access authorization.
Keep customer exports, private proposals and site admin areas behind proper login checks and access rights. A hard-to-guess URL or robots.txt rule is not enough. Listing a private path can make that path public.
New crawler rules do not delete past copies or undo past model training. To check earlier use, save evidence and find the right enquiry route. Do not promise removal of old copies without confirmation.
What differs for Malaysian and Singaporean SMEs?
Some checks overlap, but Malaysia and Singapore have separate data-protection laws. A Meta contract, robots.txt choices and checks against local law answer different questions.
Review personal data, not just public marketing copy
Malaysia’s official Personal Data Protection Act overview explains rules for the use of personal data in commercial transactions. Singapore’s PDPC overview sets out rules on how to collect, use, share and care for personal data. The Singapore page also lists cases left out of those rules, including business contact information. Do not assume that the same rule, or the same exception, applies on both sides of the border.
In either market, check staff profiles, customer reviews, enquiry forms and files people can download. Find personal data, why it is public and who agreed to post it. Public access alone does not settle all questions about later use.
Keep commercial permission and compliance separate
Meta website crawling terms do not replace checks on customer promises, third-party licences or data handling. Crawler rules cannot tell you whether a contract term can be enforced in your case. Seek qualified advice when a serious rights or compliance issue needs a legal answer.
Keep the marketing work clear and practical. Our guide to WhatsApp first-party data in Malaysia gives related help with how to sort customer details. Use that approach to keep public pages that bring in leads apart from private customer records.
This is a guide to practical steps, not legal advice. A general site review cannot establish a country-wide exemption, ban or guaranteed remedy.
What should you check before 20 November?
Write down a small site review before the planned start date. Deal first with exposed sensitive content, clear owners and tested changes, not a blanket block.
Build an evidence pack and assign owners
- Save the terms. Keep the current terms and previews, their start dates and the date you checked them. Check the live pages again before you act, as the words can change.
- Map the links. List business Pages, ad accounts, agencies, landing pages, catalogues and linked domains. Note who places orders and who can approve site changes.
- Sort the content. Keep public marketing files apart from customer data, licensed content and private files. Fix content left public by mistake first.
- Check requests. Ask your developer what crawler evidence you have and what it cannot show. Keep what you saw apart from guesses about AI training.
- Test your choices. Agree rules for each agent and check they are in place. Test key link previews, landing pages and the ad tasks that use them.
For example, a Johor shop serving Singapore customers could keep product pages public and move wholesale customer spreadsheets behind a login. Its marketing lead could ask for a focused crawler check, without assuming all bots do the same job. This illustrates a choice, not proof about that shop’s risk or a setup every SME should copy.
Put one person in charge of the record: what changed, why, who agreed and when to review it. If an agency runs your ads, make sure its steps match your site settings.
Connect technical choices to business accountability
A developer should not have to guess your rules. A marketer should not have to read server logs alone. Our guide to humans in the loop in digital marketing shows why a named person needs to oversee the work when automated systems shape business choices.
If the review finds a gap in how you run your site, explore ATELIER’s website management services. Start with the pages, controls and roles you need checked.
For help with a site review enquiry, contact Xwork.my with your domain, marketing setup and main concern. We can help clarify practical questions before you choose your next step.
Frequently asked questions
These five answers separate terms, site controls and evidence. They reflect documents reviewed on 26 October 2026, not a promise about future terms or any one site.
Is Meta definitely training AI on my business website?
No. Terms say what is allowed; crawler guides list possible uses. Neither proves your pages went into a training dataset. Even a confirmed request shows access, not a full account of later use. Check the evidence before making that claim.
When do the updated terms take effect?
Both previews show 20 November 2026. Reuters reported the move from 30 October on 23 October. But current Self-Serve Ad Terms already allow site access when you place an ad order, with research and training listed. November is not the first permission to crawl.
Will blocking Meta-ExternalAgent block every Meta crawler?
No. Meta lists agents with different roles. A Meta-ExternalAgent rule does not set rules for Meta-ExternalAds, Meta-WebIndexer or other agents. Meta also lists robots.txt exceptions for user-requested fetches and some security checks. Review and test each relevant choice.
Does robots.txt remove content already used for training?
No. Robots.txt tells crawlers your access choices. It cannot delete past copies, undo model training or secure private pages. Use login checks for private content. Look into separate enquiry or removal routes if past collection raises a specific concern.
Should my Malaysian or Singaporean SME stop all Meta advertising?
Not by default. Read the terms, check your site’s exposure and weigh business needs. Meta website crawling terms are one part of that review. Record your choice, test controls that fit the risk and get qualified legal advice for firm answers about rights or duties.
