HSMBot

HSMBot reads the public career pages of employers that the IND recognises as sponsors, so people looking for a Highly Skilled Migrant job can find their vacancies in one place. If you saw it in your logs, this page explains what it does and how to stop it.

How to recognise it

Every request carries this User-Agent:

HSMBot/1.0 (+https://hsm.arisoft.nl/bot)

What it reads, and how often

• Only the careers or vacancies pages of IND recognised sponsors, the job pages linked from them, and public job board APIs of applicant tracking systems (such as Greenhouse, Lever and Recruitee).
• The home page of each sponsor about once a month, to check that the website we link to is still the employer's own site. We keep only whether the site is online, never its text.
• At most one request every 5 seconds per website, and slower when your robots.txt sets a larger Crawl-delay.
• Usually once a day, at night (Dutch time). A job page is read when it is new, and again after about a week to catch changes.
• Never pages behind a login or forms.

What we publish

For each job: the title, the place, the date, the employment type, the salary if the posting states one, and the description as your posting gives it, with a prominent link to your original posting, where candidates apply. Email addresses and phone numbers are removed before anything is stored. We do not copy logos or images. To have your postings removed, email us; a text and data mining reservation (below) stops HSMBot altogether.

How to limit or block HSMBot

HSMBot does not follow Disallow rules in robots.txt: it reads public job postings, the way a job seeker does. To stop it, use one of these:

• A text and data mining reservation under the EU DSM Directive (article 4): a /.well-known/tdmrep.json file, a TDM-Reservation: 1 response header, or a <meta name="tdm-reservation" content="1"> tag. A reserved page is not read, and the first visit that finds the reservation also takes the jobs we already list from your site off ours.
• An email to us (below): we stop reading your site and remove your jobs.

To slow it down, robots.txt still works: HSMBot keeps a larger delay in seconds when you set one.

User-agent: HSMBot
Crawl-delay: 30

robots.txt and TDM files are cached for up to 24 hours, so a change can take a day to apply.

Remove your jobs or correct your listing

Email info@arisoft.nl with your company name. We stop reading your site, take your jobs off the site, and confirm by email, usually within two working days. Corrections to your listing are welcome at the same address.

Who runs it

Arisoft, Van Embdenstraat 820, 2628 ZP Delft, the Netherlands. KvK 42057545. Arisoft HSM is not affiliated with the IND.