Skip to content

Commit 5face68

Browse files
unclecodeunclecode
committed
cloud notice: the monthly offer, once per release, and the two moments of pain
- The offer everywhere (terminal, README, docs banner): $5 free every month, no card, try it with no signup (it was "$5 of free credit", a one-time gift). - The first-run notice returns once per minor release (the marker holds the minor version). - A crawl blocked by anti-bot protection prints "Blocked? Crawl4AI Cloud gets through the hard sites" (?ref=blocked), at most once a day per machine. - crawl4ai-setup that could not install the browser prints "Browser setup failed? Skip it" (?ref=setup). - CRAWL4AI_NO_CLOUD_NOTICE=1 still turns every line off.
1 parent b7ea000 commit 5face68

5 files changed

Lines changed: 100 additions & 32 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414

1515
---
1616
#### ☁️ Crawl4AI Cloud is live — the same crawler, hosted
17-
No browsers to run, no proxies to buy, no cookies to babysit. One key with **$5 of free credit**, and your AI agent can search, scrape, crawl and extract the web through the engine you already use. Works in Claude Code, Codex, OpenCode and Cursor with one pasted line.
17+
No browsers to run, no proxies to buy, no cookies to babysit. One key with **$5 free every month** (no card), and your AI agent can search, scrape, crawl and extract the web through the engine you already use. Works in Claude Code, Codex, OpenCode and Cursor with one pasted line.
1818

1919
👉 **[Get a key in 10 seconds](https://crawl4ai.com/?ref=readme)** · [Cloud docs](https://crawl4ai.com/docs?ref=readme)
2020
_The library stays open source, forever. The cloud is for the days you want to build fast and let us run the infrastructure._

‎crawl4ai/async_webcrawler.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -638,6 +638,13 @@ async def arun(
638638
if _blocked:
639639
crawl_result.success = False
640640
crawl_result.error_message = f"Blocked by anti-bot protection: {_block_reason}"
641+
# the moment the hosted crawler helps most; at most once a day
642+
try:
643+
from .cloud_notice import show_blocked
644+
645+
show_blocked(self.logger)
646+
except Exception:
647+
pass
641648
crawl_result.crawl_stats = _crawl_stats
642649
else:
643650
# All proxies threw exceptions and fallback either wasn't

‎crawl4ai/cloud_notice.py‎

Lines changed: 76 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,43 @@
1-
"""The Crawl4AI Cloud notice: one short block that tells a user the hosted version exists.
1+
"""The Crawl4AI Cloud notice: short lines that tell a user the hosted version exists.
22
3-
It prints in two places: at the end of ``crawl4ai-setup`` (every time), and once per machine when the
4-
first ``AsyncWebCrawler`` starts or the ``crwl`` CLI first runs (a marker file in the crawl4ai home
5-
folder remembers it). ``CRAWL4AI_NO_CLOUD_NOTICE=1`` turns it off everywhere, for operators who run the
6-
library in production logs.
3+
It prints in four places:
4+
5+
- at the end of ``crawl4ai-setup``, every time (``show_at_install``);
6+
- once per minor release per machine, when the first ``AsyncWebCrawler`` starts or the ``crwl`` CLI runs
7+
(``show_once``; a marker file in the crawl4ai home folder holds the last minor version shown);
8+
- when a crawl is blocked by anti-bot protection, at most once a day (``show_blocked``);
9+
- when ``crawl4ai-setup`` could not install the browser (``show_setup_failed``).
10+
11+
Each link carries its own ``ref`` so the cloud can tell which moment brought a user.
12+
``CRAWL4AI_NO_CLOUD_NOTICE=1`` turns every line off, for operators who run the library in production logs.
713
"""
814

915
import os
16+
import time
1017
from pathlib import Path
1118

1219
CLOUD_URL = "https://crawl4ai.com/?ref=pip"
20+
BLOCKED_URL = "https://crawl4ai.com/?ref=blocked"
21+
SETUP_URL = "https://crawl4ai.com/?ref=setup"
1322
_ENV_OFF = "CRAWL4AI_NO_CLOUD_NOTICE"
1423
_MARKER = ".cloud-notice-shown"
24+
_BLOCKED_MARKER = ".cloud-notice-blocked"
1525

1626
LINES = (
17-
"Crawl4AI Cloud is live: the same crawler, hosted. No browsers to run, no proxies, $5 of free credit.",
27+
"Crawl4AI Cloud is live: the same crawler, hosted. $5 free every month, no card. Try it now with no signup.",
1828
f"Get a key in 10 seconds: {CLOUD_URL} (the library stays open source, forever)",
1929
f"Hide this notice: {_ENV_OFF}=1",
2030
)
2131

32+
BLOCKED_LINES = (
33+
f"Blocked? Crawl4AI Cloud gets through the hard sites, $5 free every month: {BLOCKED_URL}",
34+
f"Hide this notice: {_ENV_OFF}=1",
35+
)
36+
37+
SETUP_LINES = (
38+
f"Browser setup failed? Skip it: the same crawler, hosted, nothing to install, $5 free every month: {SETUP_URL}",
39+
)
40+
2241

2342
def _home() -> Path:
2443
try:
@@ -29,6 +48,16 @@ def _home() -> Path:
2948
return Path.home() / ".crawl4ai"
3049

3150

51+
def _minor_version() -> str:
52+
"""This release's minor version, "0.9" for 0.9.3: the notice returns once per minor release."""
53+
try:
54+
from .__version__ import __version__
55+
56+
return ".".join(__version__.split(".")[:2])
57+
except Exception:
58+
return ""
59+
60+
3261
def is_off() -> bool:
3362
"""True when the operator turned the notice off with the environment variable."""
3463
return os.environ.get(_ENV_OFF, "").strip().lower() in ("1", "true", "yes")
@@ -39,35 +68,55 @@ def text() -> str:
3968
return "\n".join(LINES)
4069

4170

42-
def show_at_install(logger=None) -> None:
43-
"""Print the notice at the end of ``crawl4ai-setup``. Every run; nothing is remembered."""
44-
if is_off():
45-
return
71+
def _print(lines, logger=None) -> None:
4672
if logger is not None:
47-
for line in LINES:
73+
for line in lines:
4874
logger.info(line, tag="CLOUD")
4975
else:
50-
print(text())
76+
print("\n".join(lines))
5177

5278

53-
def show_once(logger=None) -> bool:
54-
"""Print the notice the first time the library or the CLI runs on this machine. Returns True when
55-
it printed. The marker file keeps it to one time; a machine without a writable home never repeats
56-
the notice more than once per process."""
57-
if is_off():
58-
return False
59-
marker = _home() / _MARKER
79+
def _claim(name: str, value: str) -> bool:
80+
"""True when the marker file ``name`` does not hold ``value`` yet, and writes it. A home folder that
81+
cannot be written answers True, so the line shows once per process there (rare, acceptable)."""
82+
marker = _home() / name
6083
try:
61-
if marker.exists():
84+
if marker.exists() and marker.read_text().strip() == value:
6285
return False
6386
marker.parent.mkdir(parents=True, exist_ok=True)
64-
marker.write_text("shown\n")
87+
marker.write_text(value + "\n")
6588
except Exception:
66-
# no writable home: show it this once, and again next process (rare, acceptable)
6789
pass
68-
if logger is not None:
69-
for line in LINES:
70-
logger.info(line, tag="CLOUD")
71-
else:
72-
print(text())
7390
return True
91+
92+
93+
def show_at_install(logger=None) -> None:
94+
"""Print the notice at the end of ``crawl4ai-setup``. Every run; nothing is remembered."""
95+
if is_off():
96+
return
97+
_print(LINES, logger)
98+
99+
100+
def show_once(logger=None) -> bool:
101+
"""Print the notice the first time the library or the CLI runs on this machine after a new minor
102+
release. Returns True when it printed."""
103+
if is_off() or not _claim(_MARKER, _minor_version() or "shown"):
104+
return False
105+
_print(LINES, logger)
106+
return True
107+
108+
109+
def show_blocked(logger=None) -> bool:
110+
"""Print the blocked line when a crawl hit anti-bot protection: the moment the cloud helps most.
111+
At most once a day per machine. Returns True when it printed."""
112+
if is_off() or not _claim(_BLOCKED_MARKER, time.strftime("%Y-%m-%d", time.gmtime())):
113+
return False
114+
_print(BLOCKED_LINES, logger)
115+
return True
116+
117+
118+
def show_setup_failed(logger=None) -> None:
119+
"""Print the setup line when ``crawl4ai-setup`` could not install the browser."""
120+
if is_off():
121+
return
122+
_print(SETUP_LINES, logger)

‎crawl4ai/install.py‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,16 @@ def setup_home_directory():
3939
with open(crawl4ai_config, "w") as f:
4040
f.write("")
4141

42+
def _cloud_setup_hint():
43+
"""After a failed browser install, name the hosted crawler that needs no browser."""
44+
try:
45+
from .cloud_notice import show_setup_failed
46+
47+
show_setup_failed(logger)
48+
except Exception:
49+
pass
50+
51+
4252
def post_install():
4353
"""
4454
Run all post-installation tasks.
@@ -120,11 +130,13 @@ def install_playwright():
120130
logger.warning(
121131
f"Please run '{sys.executable} -m playwright install --with-deps' manually after the installation."
122132
)
133+
_cloud_setup_hint()
123134
except Exception:
124135
# logger.error(f"Unexpected error during Playwright installation: {e}", tag="ERROR")
125136
logger.warning(
126137
f"Please run '{sys.executable} -m playwright install --with-deps' manually after the installation."
127138
)
139+
_cloud_setup_hint()
128140

129141
# Install Patchright browsers for undetected browser support
130142
logger.info("Installing Patchright browsers for undetected mode...", tag="INIT")

‎docs/md_v2/overrides/partials/cloud-banner.html‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,13 @@
1010
<div class="c4b-text">
1111
<div class="c4b-eyebrow"><span class="c4b-tag">New</span> good news for everyone who runs Crawl4AI</div>
1212
<h2 class="c4b-title">Crawl4AI <em>Cloud</em> is live.<br>The same crawler, hosted.</h2>
13-
<p class="c4b-lead">No browsers to run, no proxies to buy, no cookies to babysit. One key with <b>$5 of free credit</b>, and your AI agent can <b>search, scrape, crawl and extract</b> the web through the same engine you already use.</p>
13+
<p class="c4b-lead">No browsers to run, no proxies to buy, no cookies to babysit. One key with <b>$5 free every month</b>, and your AI agent can <b>search, scrape, crawl and extract</b> the web through the same engine you already use.</p>
1414
<p class="c4b-oss"><b>Always open source, forever.</b> The library stays here and stays free. The cloud is for the days you want to build fast and let us run the infrastructure.</p>
1515
<div class="c4b-cta">
16-
<a class="c4b-btn" href="https://crawl4ai.com/?ref=docs-banner" data-c4b-ev="banner_key_clicked">Get a key + $5 free credit →</a>
16+
<a class="c4b-btn" href="https://crawl4ai.com/?ref=docs-banner" data-c4b-ev="banner_key_clicked">Get a key, $5 free every month →</a>
1717
<a class="c4b-btn c4b-ghost" href="https://crawl4ai.com/?ref=docs-banner" data-c4b-ev="banner_cloud_clicked">See Crawl4AI Cloud</a>
1818
</div>
19-
<div class="c4b-meta"><span>no card</span><span><b>$5 free credit</b> to start</span><span>EU · Singapore · US</span><span>MCP built in</span></div>
19+
<div class="c4b-meta"><span>no card</span><span><b>$5 free</b> every month</span><span>EU · Singapore · US</span><span>MCP built in</span></div>
2020
</div>
2121
<div class="c4b-paste" aria-label="Paste this into your agent">
2222
<div class="c4b-tabs" role="tablist">
@@ -36,7 +36,7 @@ <h2 class="c4b-title">Crawl4AI <em>Cloud</em> is live.<br>The same crawler, host
3636

3737
<div class="c4b-small">
3838
<span class="c4b-star">★</span>
39-
<span class="c4b-t"><b>Crawl4AI Cloud is live</b> — the same crawler, hosted. Get a key with $5 free credit and paste one line into your agent.</span>
39+
<span class="c4b-t"><b>Crawl4AI Cloud is live</b> — the same crawler, hosted. Get a key with $5 free every month and paste one line into your agent.</span>
4040
<span class="c4b-sp"></span>
4141
<a class="c4b-go" href="https://crawl4ai.com/?ref=docs-banner" data-c4b-ev="banner_key_clicked">Get a key →</a>
4242
<button type="button" data-c4b-act="expand">show</button>

0 commit comments

Comments
 (0)