firecrawl
The context API to search, scrape, and interact with the web at scale. π₯
File Explorer
Download Latest Version (.zip)- js-sdk.yml
- publish-rust-sdk.yml
- python-sdk.yml
- rust-sdk.yml
- bug_report.md
- feature_request.md
- self_host_issue.md
- audit-ci-vuln-scan.mjs
- check_version_has_incremented.py
- eval_run.py
- requirements.txt
- resolve_api_image_version.py
- deploy-go-service.yaml
- deploy-image-staging.yml
- deploy-image.yml
- deploy-nuq-postgres.yml
- deploy-playwright.yml
- deploy-redis.yml
- eval-prod.yml
- ghcr-clean.yml
- npm-audit-claude-remediation.yml
- npm-audit.yml
- publish-dotnet-sdk.yml
- publish-elixir-sdk.yml
- publish-go-sdk.yml
- publish-java-sdk.yml
- publish-js-sdk.yml
- publish-php-sdk.yml
- publish-python-sdk.yml
- publish-ruby-sdk.yml
- publish-rust-sdk.yml
- scrape-evals.yml
- sync-skills-catalog.yml
- test-dotnet-sdk.yml
- test-go-html-to-md-service.yml
- test-go-sdk.yml
- test-java-sdk.yml
- test-js-sdk.yml
- test-php-sdk.yml
- test-ruby-sdk.yml
- test-rust-sdk.yml
- test-server.yml
- validate-lockfiles.yml
- CODEOWNERS
- dependabot.yml
- pre-commit
- concurrency_logs.sql
- config.toml
- crawler.rs
- document.rs
- engpicker.rs
- html.rs
- lib.rs
- logging.rs
- pdf.rs
- utils.rs
- .editorconfig
- .gitattributes
- .gitignore
- .prettierignore
- .taplo.toml
- .yarnrc.yml
- build.rs
- Cargo.toml
- package.json
- README.md
- rustfmt.toml
- tsconfig.json
- wasi-worker-browser.mjs
- node-fetch@2.7.0.patch
- browser.requests.http
- crawl.requests.http
- map.requests.http
- scrape.requests.http
- search.requests.http
- branding.requests.http
- sample.docx
- sample.odt
- sample.rtf
- sample.xlsx
- .gitignore
- go.mod
- go.sum
- html-to-markdown.go
- README.md
- fireclaw.test.ts
- deep-research-redis.test.ts
- index.test.ts
- index.test.ts
- index.test.ts
- v2_map.test.ts
- index.test.ts
- index.test.ts
- index.test.ts
- index.test.ts
- color-roles.test.ts
- declared-logo.test.ts
- declared-marks.test.ts
- llm-failure-fallback.test.ts
- processor-color.test.ts
- fire-engine-render-routing.test.ts
- search-query-builder.test.ts
- core.test.ts
- key-gate.test.ts
- router.test.ts
- stress.test.ts
- check-credits-middleware.test.ts
- labs.routes.test.ts
- research-keyless.routes.test.ts
- map-query-params.json
- map-redirect.json
- mocking-works-properly.json
- collect-mocks.js
- lib.ts
- scrape.test.ts
- threat-protection.test.ts
- batch-scrape.test.ts
- billing.test.ts
- concurrency.test.ts
- crawl.test.ts
- deep-research.test.ts
- deprecation.test.ts
- extract.test.ts
- iframe-selectors.test.ts
- json-extract-format.test.ts
- lib.ts
- map.test.ts
- menu.test.ts
- product.test.ts
- scrape.test.ts
- search.test.ts
- types-validation.test.ts
- webhook.test.ts
- zdr.test.ts
- agent-auth-discovery.test.ts
- audio-routing.test.ts
- batch-scrape.test.ts
- billing.test.ts
- browser-replay.test.ts
- concurrency.test.ts
- crawl-prompt.test.ts
- crawl.test.ts
- developer.test.ts
- document-converter.test.ts
- endpoint-feedback.test.ts
- express5.test.ts
- idmux.test.ts
- iframe-selectors.test.ts
- index-cache-variants.test.ts
- ip-restriction.test.ts
- key-restriction.test.ts
- keyless.test.ts
- lib.ts
- map.test.ts
- menu.test.ts
- monitor.test.ts
- parse.test.ts
- parsers-fire-pdf-async.test.ts
- parsers-pdf-page-markdown.test.ts
- parsers.test.ts
- product.test.ts
- research.test.ts
- scrape-branding.test.ts
- scrape-browser.test.ts
- scrape-cache.test.ts
- scrape-formats.test.ts
- scrape-lockdown.test.ts
- scrape-pii.test.ts
- scrape-query.test.ts
- scrape-skip-tls.test.ts
- scrape-viewport.test.ts
- scrape.test.ts
- search-feedback.test.ts
- search.test.ts
- system-prompt-rejection.test.ts
- team-siem-logging.test.ts
- team-threat-protection.test.ts
- threat-protection-billing.test.ts
- threat-protection-enforcement.test.ts
- threat-protection-zscaler.test.ts
- types-validation.test.ts
- webhook.test.ts
- wikipedia.test.ts
- zdr.test.ts
- generateDomainSplits.test.ts
- index-cache.test.ts
- lib.ts
- metadata-concat.test.ts
- wikipedia-url-parser.test.ts
- zdr-helpers.ts
- nuq-postgres.test.ts
- pdf-parser-options.test.ts
- auth.test.ts
- crawl.test.ts
- url-schema.test.ts
- acuc-cache-clear.ts
- autumn-health.ts
- check-fire-engine.ts
- concurrency-queue-backfill.ts
- crawl-monitor.ts
- index-queue-prometheus.ts
- ip-restriction-cache-clear.ts
- key-restriction-cache-clear.ts
- metrics.ts
- precrawl.ts
- redis-health.ts
- crawl-cancel.ts
- crawl-status.ts
- crawl.ts
- keyAuth.ts
- liveness.ts
- readiness.ts
- scrape.ts
- search.ts
- urlValidation.test.ts
- activity.ts
- batch-scrape.ts
- concurrency-check.ts
- crawl-cancel.ts
- crawl-errors.ts
- crawl-ongoing.ts
- crawl-status-ws.ts
- crawl-status.ts
- crawl.ts
- credit-usage-historical.ts
- credit-usage.ts
- deep-research-status.ts
- deep-research.ts
- extract-status.ts
- extract.ts
- fireclaw.ts
- generate-llmstxt-status.ts
- generate-llmstxt.ts
- map.ts
- queue-status.ts
- scrape-status.ts
- scrape.ts
- search.ts
- token-usage-historical.ts
- token-usage.ts
- types.ts
- agent-status.test.ts
- browser-billing.test.ts
- developer-access.test.ts
- redactPII-schema.test.ts
- research-timeouts.test.ts
- scrape-browser.test.ts
- search-developer-ledger.test.ts
- controller.ts
- feedback-store.ts
- internal-types.ts
- record-options.test.ts
- record-options.ts
- record.ts
- refund-policy.ts
- refund-totals.ts
- request-input.ts
- zdr-persistence.test.ts
- zdr-persistence.ts
- agent-cancel.ts
- agent-snapshot.ts
- agent-status.ts
- agent-trace.ts
- agent.ts
- batch-scrape.ts
- browser-replay.ts
- browser.ts
- concurrency-check.ts
- crawl-cancel.ts
- crawl-errors.ts
- crawl-ongoing.ts
- crawl-params-preview.ts
- crawl-status-ws.ts
- crawl-status.ts
- crawl.ts
- credit-usage-historical.ts
- credit-usage.ts
- extract-status.ts
- extract.ts
- f-search.ts
- keyless-eligibility.ts
- map.ts
- mcp-action-logs.ts
- monitor.ts
- parse-upload.ts
- parse.ts
- queue-status.ts
- research-proxy.ts
- scrape-browser.ts
- scrape-status.ts
- scrape.ts
- search-feedback.ts
- search-schema.test.ts
- search.ts
- siem-logging-schema.test.ts
- slack.ts
- support-proxy.ts
- team-siem-logging.ts
- team-threat-protection.ts
- token-usage-historical.ts
- token-usage.ts
- types.ts
- auth.ts
- index-db.ts
- index.ts
- public.ts
- auth-credit-usage-rpc.test.ts
- connection.ts
- rpc.ts
- deduplicate-obs-array.test.ts
- html-to-markdown.test.ts
- html-transformer.test.ts
- job-priority.test.ts
- keyless-usage-log.test.ts
- merge-null-val-objs.test.ts
- mix-schemas.test.ts
- spread-schema-objects.test.ts
- spur.test.ts
- transform-array-to-obj.test.ts
- url-utils.test.ts
- color-roles.ts
- declared-logo.ts
- extractHeaderHtmlChunk.ts
- llm.ts
- logo-selector.ts
- merge.ts
- processor.ts
- prompt.ts
- schema.ts
- transformer.ts
- types.ts
- deep-research-redis.ts
- deep-research-service.ts
- research-manager.ts
- cache.ts
- util.ts
- dom.ts
- selector-repair.ts
- simplify.ts
- client.ts
- prompts.ts
- anchor.ts
- generate.ts
- postprocess.ts
- validate.ts
- harness.ts
- runExtractor.ts
- config.ts
- extract.ts
- analyzeSchemaAndPrompt-f0.ts
- batchExtract-f0.ts
- checkShouldExtract-f0.ts
- singleAnswer-f0.ts
- deduplicate-objs-array-f0.ts
- merge-null-val-objs-f0.ts
- mix-schema-objs-f0.ts
- source-tracker-f0.ts
- spread-schemas-f0.ts
- transform-array-to-obj-f0.ts
- llm-cost-f0.ts
- build-document-f0.ts
- build-prompts-f0.ts
- document-scraper-f0.ts
- extraction-service-f0.ts
- llmExtract-f0.test.ts
- llmExtract-f0.ts
- reranker-f0.ts
- url-processor-f0.ts
- dereference-schema.test.ts
- source-tracker.test.ts
- deduplicate-objs-array.ts
- dereference-schema.ts
- merge-null-val-objs.ts
- mix-schema-objs.ts
- source-tracker.ts
- spread-schemas.ts
- transform-array-to-obj.ts
- model-prices.ts
- build-document.ts
- build-prompts.ts
- config.ts
- document-scraper.ts
- extract-redis.ts
- team-id-sync.ts
- types.ts
- generate-llmstxt-redis.ts
- generate-llmstxt-service.ts
- generate-llmstxt-supabase.ts
- browser-agent.ts
- browser-service-client.ts
- langsmith.test.ts
- langsmith.ts
- scrape-replay.ts
- azure-sentinel.test.ts
- azure-sentinel.ts
- event.test.ts
- event.ts
- index.test.ts
- index.ts
- metrics.ts
- store.test.ts
- store.ts
- transport.test.ts
- transport.ts
- types.ts
- canonicalize.test.ts
- canonicalize.ts
- fake-redis.ts
- store.test.ts
- store.ts
- sync.test.ts
- sync.ts
- testing.ts
- client.test.ts
- client.ts
- index.ts
- lookup-queue.ts
- sync.test.ts
- sync.ts
- testing.ts
- google-web-risk.ts
- providers.test.ts
- check-url.test.ts
- config.test.ts
- config.ts
- error.ts
- index.ts
- request.ts
- store.test.ts
- store.ts
- types.ts
- verdict.test.ts
- verdict.ts
- admin-integration-integrations-proxy.ts
- agent-auth-discovery.test.ts
- agent-auth-discovery.ts
- agent-interop.ts
- api-key-concurrency.ts
- avgrab-resolve.ts
- browser-billing.ts
- browser-session-activity.ts
- browser-sessions.ts
- canonical-url.test.ts
- canonical-url.ts
- cclog.test.ts
- cclog.ts
- change-tracking-diff.ts
- clickhouse-client.ts
- concurrency-limit.ts
- concurrency-queue-reconciler.ts
- concurrency-redis.ts
- cost-tracking.ts
- crawl-redis.test.ts
- crawl-redis.ts
- crawl-scope.ts
- custom-error.ts
- default-values.ts
- deployment.ts
- deprecations.ts
- document-formats.ts
- engpicker.ts
- entities.ts
- error-serde.ts
- error.ts
- exchange-request.ts
- exchange.test.ts
- exchange.ts
- external-request-id.ts
- fire-privacy-chunker.test.ts
- fire-privacy-chunker.ts
- fire-privacy-client.test.ts
- fire-privacy-client.ts
- format-utils.ts
- gcs-jobs.ts
- gcs-monitoring.ts
- gcs-pdf-cache.ts
- generic-ai.ts
- highlight-spans.test.ts
- highlight-spans.ts
- html-to-markdown-client.ts
- html-to-markdown.ts
- http-metrics.ts
- index-cache-metrics.ts
- ip-restriction.test.ts
- ip-restriction.ts
- job-metrics.ts
- job-priority.ts
- key-restriction.test.ts
- key-restriction.ts
- keyless-credit-projection.ts
- keyless.conversion.test.ts
- keyless.ts
- logger.ts
- map-cosine.ts
- map-utils.ts
- mcp-action-log-config.test.ts
- mcp-action-log-config.ts
- mcp-delegated-credential.ts
- native-logging.ts
- org-secret-crypto.test.ts
- org-secret-crypto.ts
- otel-tracer.ts
- owner-id.ts
- parseApi.ts
- permissions.test.ts
- permissions.ts
- permu-refactor.test.ts
- queue-full-error.ts
- ranker.test.ts
- ranker.ts
- request-origin.ts
- research-keyless.ts
- research-upstream.test.ts
- research-upstream.ts
- retry-utils.ts
- robots-txt.ts
- scrape-billing.test.ts
- scrape-billing.ts
- search-index-client.ts
- search-query-builder.ts
- spur.ts
- strings.ts
- supabase-jobs.ts
- tracking.test.ts
- tracking.ts
- url-utils.ts
- validate-country.ts
- validateUrl.test.ts
- validateUrl.ts
- withAuth.ts
- zdr-helpers.ts
- zdrcleaner.ts
- runWebScraper.ts
- admin.ts
- exchange.ts
- labs.ts
- mcp-action-logs.ts
- shared.ts
- v0.ts
- v1.ts
- v2.ts
- sitemap.ts
- shouldCheckRobots.test.ts
- index.ts
- index.ts
- brand-utils.ts
- buttons.ts
- constants.ts
- css-data.ts
- declared-marks.ts
- elements.ts
- helpers.ts
- images.ts
- index.ts
- print-script.js
- svg-utils.ts
- brandingScript.ts
- checkStatus.ts
- delete.ts
- index.ts
- scrape.ts
- index.ts
- firePDF.test.ts
- firePDFAsync.test.ts
- firePDFAsyncRouting.test.ts
- firePDFCache.test.ts
- indexPageMarkdown.test.ts
- isPdfBuffer.test.ts
- markdownToHtml.test.ts
- shadowComparison.test.ts
- async.ts
- cache.ts
- cancel.ts
- metrics.ts
- poll.ts
- result.ts
- routing.ts
- schema.ts
- submit.ts
- utils.ts
- blocks.ts
- firePDF.ts
- index.ts
- markdownToHtml.ts
- pdfParse.ts
- pdfUtils.ts
- runpodMU.ts
- shadowComparison.ts
- types.ts
- index.ts
- downloadFile.ts
- safeFetch.ts
- specialtyHandler.ts
- index.ts
- index.ts
- exchange.ts
- index.ts
- extractImages.test.ts
- extractLinks.test.ts
- rewriteUrl.test.ts
- abortManager.ts
- cacheableLookup.ts
- extractAttributes.ts
- extractImages.ts
- extractLinks.ts
- extractLinksFromMarkdown.ts
- extractMetadata.ts
- extractSmartScrape.ts
- fetch.ts
- mock.ts
- removeUnwantedElements.ts
- rewriteUrl.ts
- smartScrape.ts
- urlSpecificParams.ts
- youtube.test.ts
- index.ts
- youtube.ts
- audio.test.ts
- menu.test.ts
- product.test.ts
- sendToSearchIndex.test.ts
- video.test.ts
- agent.ts
- audio.ts
- deterministicJson.ts
- diff.ts
- index.test.ts
- index.ts
- llmExtract.test.ts
- llmExtract.ts
- menu.ts
- performAttributes.ts
- product.ts
- query.ts
- redactPII.test.ts
- redactPII.ts
- removeBase64Images.ts
- sendToSearchIndex.ts
- video.ts
- .gitignore
- error.ts
- index.ts
- README.md
- retryTracker.ts
- scrapeURL.test.ts
- shouldCheckRobots.ts
- crawler.test.ts
- dns.test.ts
- utils.test.ts
- engine-forcing.test.ts
- maxDepthUtils.test.ts
- blocklist.test.ts
- blocklist.ts
- engine-forcing.ts
- ENGINE_FORCING.md
- maxDepthUtils.ts
- crawler.ts
- sitemap.ts
- ddgsearch.ts
- fireEngine-v2.ts
- index.ts
- searxng.ts
- developer.test.ts
- developer.ts
- execute.test.ts
- execute.ts
- fireEngine.ts
- highlight-model.test.ts
- highlight-model.ts
- highlights.test.ts
- highlights.ts
- index.ts
- scrape.test.ts
- scrape.ts
- searxng.ts
- transform.ts
- slack.ts
- autumn.service.test.ts
- usage.test.ts
- autumn.service.ts
- client.ts
- firebill.ts
- types.ts
- usage.ts
- batch_billing.test.ts
- credit_billing.test.ts
- batch_billing.ts
- credit_billing.ts
- types.ts
- create.ts
- validate.ts
- index-worker.ts
- indexer-queue.ts
- client.ts
- commands.ts
- crypto.ts
- messages.ts
- oauth.ts
- redirect.ts
- signature.ts
- slack.test.ts
- store.ts
- types.ts
- data-schemas.ts
- tracking.test.ts
- tracking.ts
- log_job.test.ts
- log_job.ts
- action-logs-controller.test.ts
- action-logs-http.test.ts
- action-logs.test.ts
- action-logs.ts
- billing.ts
- cost.test.ts
- cost.ts
- dedupe.test.ts
- dedupe.ts
- judge.test.ts
- judge.ts
- persist.test.ts
- persist.ts
- run-defenses.test.ts
- run.test.ts
- run.ts
- tuning.ts
- cron.ts
- diff-orchestrator.ts
- diff.test.ts
- diff.ts
- email_recipients.test.ts
- email_recipients.ts
- email_recipients_sync.test.ts
- email_recipients_sync.ts
- interest.test.ts
- interest.ts
- jitter.ts
- judgeChange.test.ts
- judgeChange.ts
- meaningful-monitoring.test.ts
- page-events.ts
- queue.test.ts
- queue.ts
- results.ts
- runner.test.ts
- runner.ts
- scheduler.test.ts
- scheduler.ts
- stale.ts
- store-bulk-upsert.test.ts
- store.test.ts
- store.ts
- types.ts
- email_notification.ts
- monitoring_email.test.ts
- monitoring_email.ts
- monitoring_slack.test.ts
- monitoring_slack.ts
- notification-check.ts
- notification_string.ts
- worker.test.ts
- worker.ts
- enterprise-check.ts
- config.ts
- delivery.test.ts
- delivery.ts
- index.ts
- logs.ts
- queue.ts
- schema.ts
- types.ts
- client.ts
- groups.ts
- index.ts
- keyspace.ts
- ops.ts
- queue.ts
- slots.ts
- sweeper.ts
- crawl-logic.test.ts
- crawl-logic.ts
- nuq-fdb-worker.ts
- nuq-prefetch-worker.ts
- nuq-reconciler-worker.ts
- nuq-router.ts
- nuq-worker-runner.ts
- nuq-worker.ts
- nuq.ts
- redis.ts
- scrape-worker.ts
- team-semaphore.ts
- ab-test-comparison.ts
- ab-test.ts
- agent-sponsor.ts
- cclog-worker.ts
- extract-queue.ts
- extract-worker.ts
- index-cache.ts
- index-screenshot-url.test.ts
- index-screenshot-url.ts
- index.test.ts
- index.ts
- oauth-token-introspection.test.ts
- oauth-token-introspection.ts
- posthog.ts
- queue-jobs.ts
- queue-service.ts
- queue-worker.ts
- rate-limiter.test.ts
- rate-limiter.ts
- redis.ts
- redlock.ts
- sentry.test.ts
- sentry.ts
- system-monitor.ts
- zdr-worker.ts
- branding.ts
- menu.ts
- parse-diff.d.ts
- product.ts
- integration.ts
- config.ts
- harness.ts
- index.ts
- natives.ts
- types.ts
- vitest-env.d.ts
- find_uncovered_files.sh
- logview.js
- urldump-redis.js
- urldump.js
- .dockerignore
- .env.example
- .env.local
- .gitattributes
- .gitignore
- .prettierrc
- audit-ci.jsonc
- Dockerfile
- knip.config.ts
- openapi-v0.json
- openapi.json
- package.json
- pnpm-lock.yaml
- pnpm-workspace.yaml
- requests.http
- requests.kulala.http
- tsconfig.json
- v1-openapi.json
- vitest.config.ts
- AuthenticationException.cs
- FirecrawlException.cs
- JobTimeoutException.cs
- RateLimitException.cs
- ApiResponse.cs
- AuditMetadata.cs
- BatchScrapeJob.cs
- BatchScrapeOptions.cs
- BatchScrapeResponse.cs
- ConcurrencyCheck.cs
- CrawlJob.cs
- CrawlOptions.cs
- CrawlResponse.cs
- CreditUsage.cs
- Document.cs
- HighlightsFormat.cs
- JsonFormat.cs
- LocationConfig.cs
- MapData.cs
- MapOptions.cs
- MenuProfile.cs
- MonitorModels.cs
- ParseFile.cs
- ParseOptions.cs
- PdfBlocks.cs
- PdfParser.cs
- ProductProfile.cs
- QueryFormat.cs
- QuestionFormat.cs
- ResearchModels.cs
- ScrapeOptions.cs
- SearchData.cs
- SearchOptions.cs
- WebhookConfig.cs
- Firecrawl.csproj
- FirecrawlClient.cs
- FirecrawlHttpClient.cs
- ExceptionsTests.cs
- Firecrawl.Tests.csproj
- FirecrawlClientTests.cs
- ModelsTests.cs
- ParseTests.cs
- Firecrawl.sln
- README.md
- error.ex
- firecrawl.ex
- firecrawl_test.exs
- test_helper.exs
- .formatter.exs
- .gitignore
- generate.exs
- LICENSE
- mix.exs
- mix.lock
- README.md
- .dockerignore
- .gitignore
- converter.go
- docker-compose.yml
- Dockerfile
- go.mod
- go.sum
- handler.go
- handler_test.go
- main.go
- Makefile
- requests.http
- option.go
- firecrawl.go
- firecrawl_error.go
- go.mod
- go.sum
- http_client.go
- models.go
- monitor_models_test.go
- options.go
- options_test.go
- parse.go
- parse_test.go
- README.md
- version.go
- gradle-wrapper.jar
- gradle-wrapper.properties
- FirecrawlClient.java
- FirecrawlHttpClient.java
- AuthenticationException.java
- FirecrawlException.java
- JobTimeoutException.java
- RateLimitException.java
- AgentOptions.java
- AgentResponse.java
- AgentStatusResponse.java
- AuditMetadata.java
- BatchScrapeJob.java
- BatchScrapeOptions.java
- BatchScrapeResponse.java
- BrowserCreateResponse.java
- BrowserDeleteResponse.java
- BrowserExecuteResponse.java
- BrowserListResponse.java
- BrowserSession.java
- ConcurrencyCheck.java
- CrawlJob.java
- CrawlOptions.java
- CrawlResponse.java
- CreditUsage.java
- Document.java
- HighlightsFormat.java
- JsonFormat.java
- LocationConfig.java
- MapData.java
- MapOptions.java
- Menu.java
- MenuAvailability.java
- MenuImage.java
- MenuItem.java
- MenuItemIdentifiers.java
- MenuMerchant.java
- MenuPrice.java
- MenuSection.java
- Monitor.java
- MonitorCheck.java
- MonitorCheckDetail.java
- MonitorCheckPage.java
- MonitorJsonFieldDiff.java
- MonitorPageDiff.java
- MonitorPageJudgment.java
- MonitorPageSnapshot.java
- MonitorSchedule.java
- MonitorSearchTarget.java
- MonitorSearchTargetResult.java
- MonitorSummary.java
- ParseFile.java
- ParseOptions.java
- PdfBlockConfidence.java
- PdfBlockItem.java
- PdfPage.java
- PdfPageBlocks.java
- PdfParser.java
- Product.java
- ProductAvailability.java
- ProductImage.java
- ProductPrice.java
- ProductSale.java
- ProductVariant.java
- QueryFormat.java
- QuestionFormat.java
- ResearchModels.java
- ScrapeOptions.java
- SearchData.java
- SearchOptions.java
- WebhookConfig.java
- package-info.java
- AgentTest.java
- AuditMetadataTest.java
- BrowserTest.java
- CrawlTest.java
- FirecrawlClientTest.java
- FirecrawlLiveSiteTest.java
- MapTest.java
- ScrapeTest.java
- SearchOptionsTest.java
- SearchTest.java
- .gitignore
- build.gradle.kts
- gradlew
- README.md
- settings.gradle.kts
- index.test.ts
- idmux.ts
- batch.test.ts
- crawl.test.ts
- extract.test.ts
- map.test.ts
- parse.test.ts
- scrape.test.ts
- search.test.ts
- usage.test.ts
- watcher.test.ts
- monitor-job-status-retry.test.ts
- agent.test.ts
- audit-metadata.unit.test.ts
- branding.test.ts
- clientOptions.test.ts
- developer.test.ts
- errorHandler.test.ts
- extract.unit.test.ts
- menu.test.ts
- monitor-search-target.test.ts
- pagination.test.ts
- parse.unit.test.ts
- product.test.ts
- prototype-discoverability.test.ts
- research.test.ts
- scrape-browser.unit.test.ts
- scrape.unit.test.ts
- search-data-property.test.ts
- search-developer.unit.test.ts
- search-highlights.unit.test.ts
- threat-protection.unit.test.ts
- v1-aliases.test.ts
- validation.test.ts
- zodSchemaToJson.test.ts
- node-undici.d.ts
- zodSchemaToJson.ts
- index.ts
- agent.ts
- batch.ts
- browser.ts
- crawl.ts
- developer.ts
- extract.ts
- feedback.ts
- map.ts
- monitor.ts
- parse.ts
- research.ts
- scrape.ts
- search.ts
- usage.ts
- errorHandler.ts
- getVersion.ts
- httpClient.ts
- pagination.ts
- validation.ts
- client.ts
- types.ts
- watcher.ts
- index.backup.ts
- index.ts
- .env.example
- .gitignore
- audit-ci.jsonc
- jest.config.js
- LICENSE
- package.json
- pnpm-lock.yaml
- pnpm-workspace.yaml
- README.md
- tsconfig.json
- tsup.config.ts
- .env.example
- audit-ci.jsonc
- example.js
- example.ts
- example_pagination.ts
- example_v1.js
- example_v1.ts
- example_watcher.ts
- LICENSE
- package.json
- pnpm-lock.yaml
- pnpm-workspace.yaml
- tsconfig.json
- Dockerfile
- nuq.sql
- firecrawl.php
- FirecrawlClient.php
- FirecrawlHttpClient.php
- AuthenticationException.php
- FirecrawlException.php
- JobTimeoutException.php
- RateLimitException.php
- Firecrawl.php
- FirecrawlCrawl.php
- FirecrawlMap.php
- FirecrawlScrape.php
- FirecrawlSearch.php
- FirecrawlTool.php
- FirecrawlTools.php
- FirecrawlServiceProvider.php
- AgentOptions.php
- AgentResponse.php
- AgentStatusResponse.php
- AuditMetadata.php
- BatchScrapeJob.php
- BatchScrapeOptions.php
- BatchScrapeResponse.php
- BrowserCreateResponse.php
- BrowserDeleteResponse.php
- BrowserExecuteResponse.php
- BrowserListResponse.php
- BrowserSession.php
- ConcurrencyCheck.php
- CrawlJob.php
- CrawlOptions.php
- CrawlResponse.php
- CreditUsage.php
- Document.php
- HighlightsFormat.php
- JsonFormat.php
- LocationConfig.php
- MapData.php
- MapOptions.php
- Menu.php
- Monitor.php
- MonitorCheck.php
- MonitorCheckDetail.php
- ParseFile.php
- ParseOptions.php
- PDFParser.php
- Product.php
- QueryFormat.php
- QuestionFormat.php
- ScrapeOptions.php
- ScreenshotFormat.php
- SearchData.php
- SearchOptions.php
- WebhookConfig.php
- Version.php
- FirecrawlCrawlTest.php
- FirecrawlMapTest.php
- FirecrawlScrapeTest.php
- FirecrawlSearchTest.php
- FirecrawlToolsTest.php
- ModelsTest.php
- ParseTest.php
- Pest.php
- .gitignore
- CHANGELOG.md
- composer.json
- phpstan.neon.dist
- phpunit.xml.dist
- README.md
- get_error.ts
- .dockerignore
- .gitignore
- api.ts
- audit-ci.jsonc
- Dockerfile
- package.json
- pnpm-lock.yaml
- pnpm-workspace.yaml
- README.md
- tsconfig.json
- conftest.py
- test_aio_batch_scrape.py
- test_aio_crawl.py
- test_aio_extract.py
- test_aio_map.py
- test_aio_parse.py
- test_aio_scrape.py
- test_aio_search.py
- test_aio_usage.py
- test_aio_watcher.py
- .env.example
- conftest.py
- test_async.py
- test_batch_scrape.py
- test_crawl.py
- test_extract.py
- test_map.py
- test_parse.py
- test_scrape.py
- test_search.py
- test_usage.py
- test_watcher.py
- test_aio_crawl_params.py
- test_aio_crawl_request_preparation.py
- test_aio_crawl_validation.py
- test_aio_map_request_preparation.py
- test_aio_parse_request_preparation.py
- test_aio_research.py
- test_aio_scrape_request_preparation.py
- test_aio_search_request_preparation.py
- test_aio_threat_protection_request_preparation.py
- test_batch_request_preparation_async.py
- test_ensure_async.py
- test_agent.py
- test_agent_request_preparation.py
- test_agent_webhook.py
- test_audit_metadata_request_preparation.py
- test_batch_request_preparation.py
- test_branding.py
- test_crawl_params.py
- test_crawl_request_preparation.py
- test_crawl_validation.py
- test_developer_search.py
- test_map_request_preparation.py
- test_menu.py
- test_pagination.py
- test_parse_request_preparation.py
- test_product.py
- test_research.py
- test_scrape_request_preparation.py
- test_search_request_preparation.py
- test_search_response.py
- test_search_validation.py
- test_threat_protection_request_preparation.py
- test_usage_types.py
- test_webhook.py
- test_document_blocks.py
- test_metadata_extras.py
- test_metadata_extras_multivalue.py
- test_recursive_schema.py
- test_validation.py
- test_ws_watcher.py
- test_monitor_types.py
- test_recursive_schema_v1.py
- __init__.py
- client.py
- __init__.py
- agent.py
- batch.py
- browser.py
- crawl.py
- developer.py
- extract.py
- map.py
- monitor.py
- parse.py
- research.py
- scrape.py
- search.py
- usage.py
- agent.py
- batch.py
- browser.py
- crawl.py
- developer.py
- extract.py
- map.py
- monitor.py
- parse.py
- research.py
- research_docs.py
- scrape.py
- search.py
- usage.py
- __init__.py
- error_handler.py
- get_version.py
- http_client.py
- http_client_async.py
- normalize.py
- validation.py
- __init__.py
- client.py
- client_async.py
- types.py
- watcher.py
- watcher_async.py
- __init__.py
- client.py
- firecrawl.backup.py
- TODO.md
- types.py
- test_agent_integration.py
- test_api_key_handling.py
- test_change_tracking.py
- test_crawl_direct_kwargs.py
- test_format_type_defaults.py
- test_search_data_property.py
- test_timeout_conversion.py
- test_v1_aliases.py
- test_v1_async_batch_scrape.py
- .env.example
- .gitignore
- .pylintrc
- example.py
- example_aio.py
- example_pagination.py
- example_v1.py
- example_v2.py
- example_ws.py
- LICENSE
- pyproject.toml
- README.md
- requirements.txt
- setup.py
- bump_version.sh
- semver
- version.sh
- .dockerignore
- Dockerfile
- fly.toml
- Procfile
- README.md
- start-redis-server.sh
- agent_options.rb
- agent_response.rb
- agent_status_response.rb
- audit_metadata.rb
- batch_scrape_job.rb
- batch_scrape_options.rb
- batch_scrape_response.rb
- concurrency_check.rb
- crawl_job.rb
- crawl_options.rb
- crawl_response.rb
- credit_usage.rb
- document.rb
- map_data.rb
- map_options.rb
- menu_profile.rb
- monitor.rb
- parse_file.rb
- parse_options.rb
- pdf_parser.rb
- product_profile.rb
- query_format.rb
- scrape_options.rb
- search_data.rb
- search_options.rb
- client.rb
- errors.rb
- http_client.rb
- version.rb
- firecrawl.rb
- client_test.rb
- test_helper.rb
- .gitignore
- firecrawl-sdk.gemspec
- Gemfile
- LICENSE
- Rakefile
- README.md
- example.rs
- agent.rs
- audit_metadata.rs
- batch_scrape.rs
- client.rs
- crawl.rs
- error.rs
- lib.rs
- map.rs
- monitor.rs
- parse.rs
- research.rs
- scrape.rs
- search.rs
- serde_helpers.rs
- types.rs
- .env.example
- audit_metadata.rs
- v2_e2e.rs
- .gitignore
- Cargo.lock
- Cargo.toml
- CHANGELOG.md
- README.md
- azuredeploy.json
- connectorDefinition.json
- dataConnector.json
- DCR.json
- README.md
- sample-event.json
- atkinson-bold.woff
- atkinson-regular.woff
- declared-logo.png
- example-long.pdf
- example.json
- example.pdf
- favicon-declared-180.png
- favicon.png
- firecrawl-logo.png
- firecrawl-wordmark.png
- llms-underscore.txt
- changeTracking.jpg
- lw3-d2-3.webp
- lw3-d3-2.webp
- lw3-d4-2.webp
- lw3-d5-2.webp
- lw3-d5-3.webp
- lw3-d6-2.webp
- lw3-d7-2.webp
- or_firecrawl.webp
- search-endpoint.jpg
- series-a.webp
- templateslaunch.webp
- blog-placeholder.jpg
- firecrawl-app-icon.png
- firecrawl-icon.png
- firecrawl-light-logo.png
- firecrawl-light-logo.svg
- firecrawl-light-wordmark.png
- firecrawl-light-wordmark.svg
- firecrawl-logo.png
- firecrawl-logo.svg
- firecrawl-wordmark.png
- firecrawl-wordmark.svg
- BaseHead.astro
- Footer.astro
- FormattedDate.astro
- Header.astro
- HeaderLink.astro
- firecrawl-v2-series-a-announcement.md
- introducing-firecrawl-templates.md
- introducing-search-endpoint.md
- launch-week-iii-day-1-introducing-change-tracking.md
- launch-week-iii-day-2-announcing-fire-1.md
- launch-week-iii-day-3-extract-v2.md
- launch-week-iii-day-4-announcing-llmstxt-new.md
- launch-week-iii-day-5-dev-day.md
- launch-week-iii-day-6-firecrawl-mcp.md
- launch-week-iii-day-7-integrations.md
- open-researcher-interleaved-thinking.md
- unicode-post.md
- BlogPost.astro
- index.astro
- [...slug].astro
- [...slug].astro
- index.astro
- about.astro
- branding-declared-only.astro
- code-block.astro
- index.astro
- product.astro
- robots.txt.ts
- global.css
- consts.ts
- content.config.ts
- .gitignore
- .npmrc
- .prettierrc
- astro.config.mjs
- audit-ci.jsonc
- package.json
- pnpm-lock.yaml
- pnpm-workspace.yaml
- README.md
- tsconfig.json
- crawl.json
- scrape.json
- run.ipynb
- CPU-utilization-report-test-1.png
- memory-utilization-report-test-1.png
- metrics-test-2.png
- metrics-test-3.png
- metrics-test-4.png
- metrics-test-5.png
- test-run-report.json
- load-test-1.md
- load-test-2.md
- load-test-3.md
- load-test-4.md
- load-test-5.md
- metrics-fire-engine-test-7-2.png
- metrics-fire-engine-test-7.png
- metrics-fire-engine-test-8.png
- metrics-test-6.png
- metrics-test-7.png
- metrics-test-8.png
- load-test-6.md
- load-test-7.md
- load-test-8.md
- .env.example
- audit-ci.jsonc
- jest.config.js
- jest.setup.js
- load-test.yml
- package.json
- pnpm-lock.yaml
- pnpm-workspace.yaml
- README.md
- favicon.ico
- vite.svg
- button.tsx
- card.tsx
- checkbox.tsx
- collapsible.tsx
- input.tsx
- label.tsx
- radio-group.tsx
- ingestion.tsx
- ingestionV1.tsx
- utils.ts
- App.tsx
- index.css
- main.tsx
- vite-env.d.ts
- .gitignore
- audit-ci.jsonc
- components.json
- eslint.config.js
- index.html
- LICENSE
- package.json
- pnpm-lock.yaml
- pnpm-workspace.yaml
- postcss.config.js
- README.md
- tailwind.config.js
- tsconfig.app.json
- tsconfig.json
- tsconfig.node.json
- vite.config.ts
- README.md
- README.md
- actions.png
- alert.png
- discord.png
- finished.png
- linechart.png
- new-alert.png
- new-server.png
- sneak-peek.png
- supabase_connect.png
- webhook.png
- notebook.ipynb
- notebook.md
- heroku-scheduler.png
- meme.png
- pa-console.png
- pa-scheduler.png
- ph-homepage.png
- ph-sample.png
- workflow.png
- notebook.ipynb
- notebook.md
- cron-syntax.png
- notebook.ipynb
- notebook.md
- mastering-map-endpoint.ipynb
- mastering-map-endpoint.md
- mastering-scrape-endpoint.ipynb
- mastering-scrape-endpoint.md
- mastering-the-crawl-endpoint.ipynb
- actions.png
- finished_actions.png
- hacker_news_homepage.png
- output.png
- async_scheduler.py
- bs4_scraper.py
- cron_scraper.py
- firecrawl_scraper.py
- scrape_scheduler.py
- notebook.ipynb
- notebook.md
- claude-3.7-stock-analyzer.py
- claude3.7-web-crawler.py
- claude-3.7-web-extractor.py
- claude_stock_analyzer.py
- web-data-contradiction-testing-using-llms.mdx
- crm_lead_enrichment.py
- .env.example
- apartment_finder.py
- README.md
- requirements.txt
- .gitignore
- deepseek-v3-extract.py
- README.md
- requirements.txt
- .gitignore
- deepseek-v3-crawler.py
- README.md
- requirements.txt
- find_internal_link_opportunites.ipynb
- README.md
- gemini-2.0-crawler.py
- gemini-2.0-web-extractor.py
- .env.example
- gemini-2.5-crawler.py
- README.md
- requirements.txt
- .env.example
- cli.py
- README.md
- requirements.txt
- .env.example
- .gitignore
- gemini-2.5-web-extractor.py
- README.md
- requirements.txt
- gemini-github-analyzer.py
- .env.example
- .gitignore
- gpt-4.1-company-researcher.py
- README.md
- requirements.txt
- .env.example
- .gitignore
- gpt-4.1-web-crawler.py
- README.md
- requirements.txt
- gpt-4.5-crawler.py
- grok_web_crawler.py
- groq_website_analyzer.py
- requirements.txt
- bs4_scraper.py
- firecrawl_scraper.py
- requirements.txt
- haiku_web_crawler.py
- internal_link_assistant.py
- job-resources-analyzer.py
- api.yaml
- configmap.yaml
- nuq-postgres.yaml
- nuq-worker.yaml
- playwright-service.yaml
- README.md
- redis.yaml
- secret.yaml
- worker.yaml
- values.yaml
- values.yaml
- _helpers.tpl
- cclog-worker-deployment.yaml
- configmap.yaml
- deployment.yaml
- extract-worker-deployment.yaml
- nuq-postgres-deployment.yaml
- nuq-postgres-pvc.yaml
- nuq-prefetch-worker-deployment.yaml
- nuq-worker-deployment.yaml
- playwright-configmap.yaml
- playwright-deployment.yaml
- playwright-service.yaml
- rabbitmq-deployment.yaml
- redis-deployment.yaml
- redis-service.yaml
- secret.yaml
- service.yaml
- worker-deployment.yaml
- .helmignore
- Chart.yaml
- README.md
- values.yaml
- .env.example
- .gitignore
- llama4-maverick-web-crawler.py
- README.md
- requirements.txt
- .env.example
- .gitignore
- llama-4-maverick-extractor.py
- README.md
- requirements.txt
- mistral-small-3.1-crawler.py
- mistral-small-3.1-extractor.py
- o1_job_recommender.py
- o1_web_crawler.py
- o1_web_extractor.py
- o3-mini-deal-finder.py
- o3-mini_company_researcher.py
- o3-mini_web_crawler.py
- .env.example
- .gitignore
- o3-web-crawler.py
- README.md
- requirements.txt
- .env.example
- .gitignore
- o4-mini-web-crawler.py
- README.md
- requirements.txt
- README.md
- .env.example
- main.py
- README.md
- requirements.txt
- .env.example
- main.py
- requirements.txt
- r1_company_researcher.py
- R1_web_crawler.py
- .env.example
- app.py
- requirements.txt
- .env.template
- .prettierignore
- airbnb_listings.json
- airbnb_prices_chart.png
- codeInterpreter.ts
- index.ts
- model.ts
- package-lock.json
- package.json
- prettier.config.mjs
- README.md
- scraping.ts
- simple_web_data_extraction_with_claude.ipynb
- sonnet_web_crawler.py
- turning_docs_into_api_specs.py
- claude-visualize-website-topics.ipynb
- web-data-extraction-using-llms.mdx
- web-data-rag--with-llama3.mdx
- website_qa_with_gemini_caching.ipynb
- website_qa_with_gemini_flash_caching.ipynb
- attributes-extraction-js-sdk.js
- attributes-extraction-python-sdk.py
- README.md
- README.md
- README.md
- README.md
- firecrawl_logo.png
- firecrawl_logo_v1.png
- open-source-cloud.png
- auth-and-env.md
- endpoint-selection.md
- integration-patterns.md
- project-intake.md
- sdk-installation.md
- verification.md
- SKILL.md
- SKILL.md
- auth-flow.md
- project-setup.md
- sdk-installation.md
- SKILL.md
- freshness-and-liveness.md
- SKILL.md
- SKILL.md
- README.md
- .gitattributes
- .gitignore
- .gitmodules
- AGENTS.md
- CLAUDE.md
- CONTRIBUTING.md
- docker-compose.yaml
- LICENSE
- README.md
- SELF_HOST.md
# Installation Guide
1. Get the code
git clone https://github.com/firecrawl/firecrawl
Downloads the entire project code from GitHub to your computer.
cd firecrawl
Moves into the project folder you just downloaded.
2. Official Install Script
Easy Recommendedpip install firecrawl-py
Installs the package published on PyPI directly β no need to clone the source.
gem install firecrawl-sdk
Installs the package published on RubyGems directly.
After installing, open a new terminal and run the program's version command (e.g. --version) to confirm it worked.
Pulled directly from this repo's README.
3. Docker
EasyPrerequisites
- Git Needed to download the project code from GitHub.
- Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
docker compose -f apps/go-html-to-md-service/docker-compose.yml up -d --build
Runs the command against the services defined in the compose file.
Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.
4. .NET
MediumPrerequisites
dotnet add package firecrawl-sdk
Type this command into your terminal and run it.
After dotnet run, check the message or address shown in the terminal.
Pulled directly from this repo's README.
5. Gradle (Java/Kotlin)
MediumPrerequisites
- Git Needed to download the project code from GitHub.
- JDK (Java) Required to build and run Java/Kotlin projects.
- Gradle If the repo includes gradlew (Gradle Wrapper), you don't need to install Gradle separately.
β οΈ This is a large repository, so this method may point to an internal sub-package rather than the actual core product. Check the full README as well.
cd apps/java-sdk
This project's files live in a subfolder, so move into it first.
./gradlew build
Builds the project using Gradle.
A BUILD SUCCESSFUL message means it worked. The output is created under build/.
6. Node.js
EasyPrerequisites
npm install firecrawl
Downloads and installs the libraries listed in package.json.
After running the command, open the address shown in the terminal (usually something like http://localhost:3000) in your browser.
Pulled directly from this repo's README.
7. Python
EasyPrerequisites
pip install firecrawl-py
Installs the package published on PyPI directly β no need to clone the source.
If it runs without errors and prints output in the terminal, it worked.
Pulled directly from this repo's README.
8. PHP (Composer)
EasyPrerequisites
composer require firecrawl/firecrawl-sdk
Type this command into your terminal and run it.
Run it with a built-in server like php -S localhost:8000, then open http://localhost:8000 in your browser.
Pulled directly from this repo's README.
9. Go
MediumPrerequisites
β οΈ This is a large repository, so this method may point to an internal sub-package rather than the actual core product. Check the full README as well.
cd apps/api/sharedLibs/go-html-to-md
This project's files live in a subfolder, so move into it first.
go build ./...
Compiles the Go program into an executable.
go run .
Runs the Go program directly without a separate build step.
If the build finishes without errors, it worked. If you used go run ., check the terminal output.
10. Rust
MediumPrerequisites
- Git Needed to download the project code from GitHub.
- Rust (rustup) Installing via rustup also installs cargo.
β οΈ This is a large repository, so this method may point to an internal sub-package rather than the actual core product. Check the full README as well.
cd apps/api/native
This project's files live in a subfolder, so move into it first.
cargo build --release
Compiles the Rust project.
cargo run
Builds and then immediately runs the program.
If cargo build finishes without errors, it worked. The executable is created under target/.
11. Ruby
EasyPrerequisites
β οΈ This is a large repository, so this method may point to an internal sub-package rather than the actual core product. Check the full README as well.
cd apps/ruby-sdk
This project's files live in a subfolder, so move into it first.
bundle install
Installs the Ruby libraries listed in the Gemfile.
If bundle install finishes without errors, continue with the run command from the README (e.g. rails server).
12. Make
MediumPrerequisites
- Git Needed to download the project code from GitHub.
- Make Usually pre-installed on Linux/macOS. On Windows, install separately (e.g. via MSYS2 or WSL).
β οΈ This is a large repository, so this method may point to an internal sub-package rather than the actual core product. Check the full README as well.
cd apps/go-html-to-md-service
This project's files live in a subfolder, so move into it first.
make
Compiles the code based on the generated build configuration to produce an executable.
If it finishes without errors, it worked. Try running the generated executable directly.
// repository documentation
Was this content helpful?
(0 ratings)