BirdPitch is an X (Twitter) outreach tool. Its crawler identifies itself as BirdPitch/1.0 (+https://birdpitch.com/bot).
What it fetches, and when
The crawler fetches a single page: the homepage of a website, at the moment the owner of that website pastes the URL into BirdPitch during onboarding. It reads the title, meta description, headings and visible text to draft a business profile, and it does not crawl further, follow internal links, or return on a schedule.
It is therefore user-initiated rather than continuous. If you see it in your logs, somebody entered your site's address into BirdPitch, and in almost every case that is you or a colleague signing up.
Verifying that a request is genuinely from BirdPitch
A User-Agent string can be set by anyone, so we also sign the request. BirdPitch supports Web Bot Auth: signed requests carry Signature-Agent, Signature-Input and Signature headers as described in RFC 9421, using Ed25519.
The Signature-Agent header points at https://birdpitch.com/.well-known/http-message-signatures-directory, a JWKS containing our public key. Fetch it, select the key whose kid matches the keyid in Signature-Input, and verify the signature over the covered components. A request that fails verification is not from us, whatever its User-Agent claims.
Requests made through a headless browser fall back to being identified by User-Agent only, because redirects there are followed internally and a signature minted for the original host would not verify at the redirected one.
Blocking it
The crawler honours robots.txt. To refuse it entirely:
User-agent: BirdPitch
Disallow: /Blocking has one visible consequence: a customer entering your site into BirdPitch gets a profile drafted from their own description instead of your page's content. Nothing else about the product depends on this crawler.
Contact
Questions about crawler behaviour, or a request to be excluded, can go through the contact page.