JoliStudioBot, our crawler

You spotted JoliStudioBot in your server logs and want to know what it was doing there? This page is for you. It’s joli.studio’s crawler: it doesn’t roam the web on its own, it reads a website when a person asks for it, and only that one.

When it comes by

There are two doors, and no more:

• The brand analysis, in the app. Someone pastes their website’s address on a project’s Brand page: the crawler reads the page to pull out the name, the tagline, the colors, the fonts, the tone and the socials (see Analyze your brand from your website).
• The wall, on wall.joli.studio. Someone pastes a website address to get nine posts in its brand: the crawler reads the site the same way.

How to recognize it

It comes by in two ways, always under its name.

Reading the page and its stylesheets announces itself with this User-Agent:

Mozilla/5.0 (compatible; JoliStudioBot/1.0; +https://joli.studio/guide/jolistudiobot/)

It also looks at the home page in a browser, the way a visitor does: a headless Chrome on Cloudflare’s network, whose regular User-Agent ends with JoliStudioBot/1.0 (+https://joli.studio/guide/jolistudiobot/). That is how it sees the page’s real colors, real fonts and photos instead of guessing them from the code.

What it reads, and what it doesn’t do

• It reads a few public pages: the site’s home page, sometimes a few linked pages, and their stylesheets for the colors and fonts. A handful of requests each time, no more.
• In the browser, the home page loads as it would for a visitor, with its images, fonts and scripts. It takes two screenshots of the top of the page, used only for the analysis at hand and never kept.
• It doesn’t sign in anywhere, fills in no forms and accepts no cookies: it hides the consent banner instead of answering it, and keeps nothing from one visit to the next.
• It refuses anything that isn’t public: local addresses and private networks.
• What it reads serves the request at hand, not training a model.

robots.txt, respected

For the wall, the crawler reads the site’s robots.txt first. If it’s disallowed there, it doesn’t read the site, doesn’t open it in the browser, and the wall isn’t made.

In the app, it’s the person who asks for the analysis of their own website, to fill in their brand profile: the reading starts from their request.

If something looks wrong

1. Note what you see

   The time, the address visited and the request’s User-Agent, as your logs show them.

2. Write to us

   At bonjour@joli.studio: we look into it, and we reply.