Renable downloads two things from your website: your product feed, on every import, and your product images, so it can process them and show them in your ads. Both downloads look like automated traffic to your website, because they are. If your website, server or CDN runs bot protection, it may refuse them even though the same addresses open fine in your browser. The fix is to give Renable a User-Agent of your own choosing and have whoever manages your website allow it.
This article covers how to recognise the block, how to set the User-Agent, where the allow setting usually lives for the common CDNs and firewalls, and what to say to your IT department or agency, including why Renable can't give you an IP address to allow instead.
How to tell it's a block#
The symptoms differ by what's being blocked:
- The feed import fails. The catalog's Activity tab shows a failed run with the notice "The feed's server refused the request (HTTP 403 Forbidden) without asking for a sign-in", while the feed address opens in a private browser window. See My feed import failed for the other causes a failed import can have.
- Images stop downloading. The catalog's Notes and events section and your dashboard show the error Your image host is blocking Renable's downloads, naming the host and how many images it refused. An individual image shows a warning beginning "The image host blocked the download as automated traffic". The full list of image warnings, and which of them this fix applies to, is under Why can't my images be retrieved?.
Both can happen at once, and both have the same fix. The usual cause is a CDN's bot and scraper protection, which treats any download not made by a person's browser as something to refuse; the feed and the images are blocked for the same reason.
Why a User-Agent, and not an IP address#
Every request Renable makes carries a User-Agent header, the short text that identifies the software asking. Browsers send one, and so do search-engine crawlers and tools like Renable. Unless you set one, Renable sends a header that looks like an ordinary desktop browser, which many hosts accept as is; bot protection that inspects more than the header will still spot that no browser is behind it.
A custom User-Agent turns Renable's downloads into traffic your host can recognise by name and allow on purpose. Every download Renable makes from your website sends it: the scheduled feed import, the live preview when you edit the feed source, the download button on the Data sources tab, and every image download.
Renable does not have fixed outgoing IP addresses, and it won't offer them. Renable's downloads run on cloud infrastructure that scales up and down as the work demands, and each download leaves from whichever address the cloud provider assigns at that moment. There is no list to give you, and the alternative, allowing the cloud provider's entire address range, would also let through everyone else who runs on that cloud, which is not something to ask a security team for. Allowing a User-Agent is the standard way to let a known service through a bot filter, it's what the vendors below document, and it's the only setting Renable supports.
If someone insists on an IP address, the honest answer is: Renable identifies itself by User-Agent, not by address, and the allow rule should match that header.
Step 1: set the User-Agent on the catalog#
-
Open the catalog from your project's Catalogs section and go to its Settings tab.
-
Expand Advanced settings and fill in User-Agent header for crawling feeds and images. We recommend this value:
Mozilla/5.0 (compatible; Renable/1.0; +https://renable.com)
It follows the convention well-behaved crawlers use: a name, a version and a web address that says who's asking. You can pick any text, but keep it distinctive, so the allow rule on your side matches nothing else.
-
Publish the catalog. The setting is a catalog setting, so it applies to the catalog's product feed, any auxiliary feeds and all of its images. A catalog with several feeds has one User-Agent.
You can also ask Aimée: "Set this catalog's User-Agent to Mozilla/5.0 (compatible; Renable/1.0; +https://renable.com)". She changes the setting and publishes once you approve.
If the block only affects images, you can still set the User-Agent first and confirm it's being sent: use Refetch image now on a blocked image before and after the allow rule goes in. The error message on the catalog quotes the User-Agent in use, so you can check what your IT department needs to match.
Step 2: have it allowed where the blocking happens#
Send the exact User-Agent string to whoever manages your website or CDN. What they need to do is add an allow (sometimes called whitelist, skip, bypass or exception) rule for requests whose User-Agent header contains that text, placed so it runs before the bot protection. Where that rule lives depends on the product. The pointers below are a starting point; vendors move these menus around, so treat the names as what to search for in their documentation rather than exact paths.
Cloudflare#
Like every CDN below, Cloudflare blocks through its bot and scraper protection, and the exception has to be made in that feature rather than in a general firewall rule.
- On the Free plan the feature is Bot Fight Mode, and Cloudflare documents that it cannot be bypassed with WAF custom rules or Page Rules: the Skip, Bypass and Allow actions have no effect on it. The choices are to turn Bot Fight Mode off, or to move to a plan with Super Bot Fight Mode, which supports exceptions.
- On Pro, Business and Enterprise the feature is Super Bot Fight Mode, and Cloudflare documents the exception mechanism: create a WAF custom rule with the Skip action, since custom rules run before it. The rule's expression matches the User-Agent header (for example User Agent contains "Renable"), and the Skip action should include the Super Bot Fight Mode rules. In the dashboard this is under Security, then WAF, then Custom rules.
- Check the other security features too if downloads still fail after the rule is in: a WAF managed rule or a rate limiting rule can block separately, and the Browser Integrity Check can reject requests whose headers it considers suspicious. The Cloudflare security events log shows which feature refused a request and why, which is the quickest way to find the right switch.
AWS CloudFront and AWS WAF#
An image host on CloudFront is usually blocked by AWS WAF, often through the Bot Control managed rule group. AWS documents the exception for a specific user agent: set the rule that signals non-browser user agents to Count, then add a rule that acts only on requests that carry its label and don't match the User-Agent you want to allow (the worked example is at Bot Control example: creating an exception for a blocked user agent). The simpler alternative is an allow rule on the User-Agent header placed earlier in the web ACL than the bot rules. The WAF logs show which rule blocked a request, which tells your team whether it was Bot Control or something else.
Akamai#
Akamai's bot protection is Bot Manager. It can define a custom-defined bot, identified by its User-Agent and optionally other request details, and place it in a custom bot category whose action is Allow in the security policy. This is configured in Akamai Control Center under the property's security configuration, usually by the agency or Akamai contact who manages it, so the thing to ask for is "a custom-defined bot for this User-Agent, in a category set to allow".
Fastly#
Fastly's Next-Gen WAF uses request rules: a rule whose condition matches the User-Agent header, with the Allow action. Fastly documents that an allow rule takes precedence over a block rule, so it doesn't need to come first. If the site uses a Fastly VCL service without the WAF, a block is in the site's own VCL and whoever wrote it needs to add the exception there.
Imperva, Sucuri and other hosted firewalls#
Hosted firewalls such as Imperva (Incapsula) and Sucuri have an allow or whitelist section in their dashboards. Some offer a User-Agent condition there and some only allow by IP address, which doesn't help with Renable; where only IP allowing exists, the practical options are to turn the bot-blocking rule down for the image path, or to ask the vendor's support how to except a user agent. Ask whoever manages the firewall to look for an allow rule on the User-Agent header first.
Your own server or a WordPress plugin#
If there's no CDN in front, the block is on your server: a rule in nginx or Apache, a hosting provider's own bot filter, or a WordPress security plugin (Wordfence, Sucuri's plugin and similar), each with its own allowlist. The same request applies: allow requests whose User-Agent contains your chosen text. Two related settings often sit next to it:
- Hotlink protection blocks image requests that come from another website. Renable sends your store's own address as the referrer on image downloads, so a plain hotlink rule normally lets them through, but a strict one that requires an exact page may not.
- Rate limiting isn't a block: Renable slows down when your host asks it to, and the catalog shows "The image host asked us to slow down" instead. Those images download again on their own.
Store platforms#
If your images are served from your store platform's own image CDN (a Shopify, Wix or Squarespace image address, for example), its bot protection isn't yours to configure. Check first whether the blocked host is your own domain or one your agency set up, which is where an allow rule can be added; if the refused host really is the platform's, the question goes to the platform's support.
What to send your IT department#
A short, complete request saves a round trip:
Renable (our product feed and ad tool) downloads our product feed and product images, and our bot protection is refusing it. Please add an allow rule for requests whose User-Agent header contains Renable/1.0 (the full header is Mozilla/5.0 (compatible; Renable/1.0; +https://renable.com)), applied to the product feed address and the image domain, placed before the bot-protection rules. Renable doesn't have fixed IP addresses, so the rule needs to match the User-Agent rather than an address.
Replace the header with the one you actually set, and name the feed address and image domain so they know which site or CDN configuration to change.
Step 3: confirm it works#
Once the rule is in:
- For the feed: click the download button on the feed's card on the Data sources tab. It fetches the feed exactly as an import does, custom User-Agent included, and saves what your server sends. A file of products means you're through; then use Import now in the health strip to run the import.
- For images: use Refetch image now on one of the blocked images. If it downloads, the rule works. You don't need to click through the rest: while the block error is open, Renable tries a few of the refused images every hour with the catalog's current User-Agent, and as soon as one goes through it downloads all of them again and the error clears on its own, usually within three hours of your fix.
If the refetch still fails, the error message tells you how the host answered. A plain refusal with no sign of bot protection can be a hotlink rule or an expired image address rather than the bot filter; the warnings are explained under Why can't my images be retrieved?.
Or ask Aimée#
Aimée can set the catalog's User-Agent for you, re-run the import once your host allows it, and read the run's warnings to tell you whether the block is gone. She can't change your CDN, so the allow rule is the one step that stays with you or your IT department.