visibian.HQ
visibian.com

PerplexityChecked 29 Oct 2026

Perplexity-User

Opens a page when a Perplexity user asks a question that needs it.

Run by
Perplexity
Used for
When a user asks
Follows robots.txt
Not always
Shut out by
9% of top websites

What blocking it changes

It may still open a page a user asks for.

"Since a user requested the fetch, this fetcher generally ignores robots.txt rules."

Perplexity, read 29 Oct 2026
Figure 1How many shut out Perplexity-User while letting Googlebot in
Most visited websites9%59 of 669
S&P 500 companies1%3 of 410
From our two studies of 29 October 2026: the 669 most visited websites and the 410 S&P 500 companies whose robots.txt we could read. 60 of the websites and 13 of the companies name Perplexity-User in the file. The website study, the S&P 500 study.

The lines to copy

To shut it out
User-agent: Perplexity-User
Disallow: /
To let it in
User-agent: Perplexity-User
Allow: /

A crawler follows only the lines under the User-agent line that names it, and the lines under User-agent: * only when no line names it. So lines for one crawler do not change the rules for the others.

Perplexity publishes the addresses it crawls from, so a request can be checked against them: the list.

More from Perplexity

  • PerplexityBot Finds and links pages for Perplexity's search results. Perplexity says it is not used to train AI models.