BankalyzeBot

What BankalyzeBot is

BankalyzeBot is a web crawler operated by Bankalyze. It monitors the public websites of Danish banks for changes, such as new news and announcement pages, for the Bankalyze service.

User-Agent

Every request identifies itself with this User-Agent header:

BankalyzeBot/1.0 (+https://go.bankalyze.com/bot)

The robots.txt token is BankalyzeBot.

What it fetches and how often

Bank websites, once a day at 04:30 UTC:

• For each monitored bank domain it fetches /robots.txt, then the XML sitemaps that robots.txt lists (or /sitemap.xml when none are listed), provided robots.txt permits them.
• It compares the list of page URLs with the previous run. For news pages that are new since the previous run, it fetches at most 10 newly listed news pages per bank to read their title and description, newest first, skipping any page robots.txt disallows. Each of these requests times out after 10 seconds.
• Sitemap index files are followed up to two levels deep and at most 50 child sitemaps. Only URLs on the monitored domain are requested; redirects to other domains are not followed.
• Banks are processed one after another with a pause of 5 seconds between banks and between page requests, and each sitemap request times out after 30 seconds.

Job postings, once a day at 07:00 UTC:

• It reads the job boards that the banks publish through their recruitment systems (such as Oracle, Workday and JobMatchProfile), with a pause of 2 seconds between boards and between the pages of a board.
• It reads each recruitment system's robots.txt first and skips any board robots.txt disallows. Only a posting's id, title, link and location are kept.

Rulings of Det Finansielle Ankenævn (fanke.dk), once a day at 06:40 UTC:

• It reads the listing pages of each board at www.fanke.dk, and the list of rulings an institution will not follow, and fetches the page of each ruling it has not seen before, at most 150 per run.
• It reads robots.txt first and skips any page robots.txt disallows. Requests are spaced by 1 second between pages.

robots.txt compliance

BankalyzeBot reads your robots.txt before every crawl and obeys the rules for the BankalyzeBot token and for *. To block it entirely, add:

User-agent: BankalyzeBot
Disallow: /

Feeds and status APIs

Bankalyze also reads a few published machine endpoints, such as news RSS feeds and the JSON API of status pages. These are single documented endpoints read as a client, not crawled, so they do not use the BankalyzeBot identity. They identify themselves with this User-Agent header instead:

Bankalyze/1.0 (+https://go.bankalyze.com/bot)

Source IP address

Requests come from 5.75.215.33 (IPv4 only).

Contact

Questions or concerns about BankalyzeBot: [email protected]