{"repo":"MontFerret/ferret","free":true,"listed":false,"github":"https://github.com/MontFerret/ferret","clone":"git clone https://github.com/MontFerret/ferret.git","description":"Declarative data automation language and Go runtime for structured extraction workflows.","language":"Go","stars":6008,"topics":["browser-automation","chrome-devtools-protocol","data-automation","data-extraction","dsl","go","golang","golang-library","html","library","query-language","runtime","web-crawling","web-scraping"],"license":"Apache-2.0","category":"data_automation","readme_excerpt":"# Ferret\n

\n\t\n\t\t\"Build\n\t\n\t\n\t\t\"Mastodon\n\t\n\t\n\t\t\"Telegram\n\t\n\t\n\t\t\"Ferret\n\t\n\t\n\t\t\"Apache-2.0\n\t\n

\n\n

\n\t\n\t\tTry it!\n\t\n\t\n\t\tDocs\n\t\n\t\n\t\tCLI\n\t\n\t\n\t\tTest runner\n\t\n\t\n\t\tWeb worker\n\t\n

\n\n---\n\n## Explore Ferret v2\n\nFerret v2 is currently in alpha. You can try the new syntax in the playground and read more about the design behind the new runtime and language capabilities:\n\n- [Try Ferret v2 in the Playground](https://ferretlang.org/try/)\n- [On the Road to Ferret v2](https://ferretlang.org/blog/ferret-v2-announcement/)\n- [Inside Ferret v2: The New Execution Model](https://ferretlang.org/blog/ferret-v2-execution-model/)\n- [Inside Ferret v2: New Language Capabilities](https://ferretlang.org/blog/ferret-v2-new-syntax/)\n\n---\n\n> **Notice:** This branch contains the upcoming **Ferret v2**. For the stable v1 release, please visit [Ferret v1](https://github.com/MontFerret/ferret/tree/v1).\n\n---\n\n## What is it?\n\nFerret is a declarative-first, expression-oriented embedded language and runtime for data automation.\n\nFQL combines querying, transformation, synchronization, and structured results with host-defined values and capabilities. Applications can embed the runtime and decide exactly which functions, modules, data, and external operations a program can use.\n\nThe language keeps a declarative core and adds domain-oriented orchestration plus constrained mutable state for automation that cannot be expressed as a pure data transformation. It is deliberately focused rather than a general-purpose scripting language.\n\n### Features\n\n- Purpose-built declarative language for querying, transforming, synchronizing, and automating structured data\n- Embeddable Go runtime with reusable compiled plans and isolated execution sessions\n- Capability-based host values for exposing application objects, resources, and external systems directly to FQL\n- Unified query model for browsers, APIs, databases, documents, and custom data sources\n- Extensible runtime through namespaced functions, modules, hooks, and custom value types\n- Event-driven synchronization and dispatch for interacting with asynchronous and stateful resources\n- Managed resource lifecycle for files, connections, cursors, streams, and other host resources\n- Bytecode VM and portable programs for efficient repeated execution and precompiled artifacts\n\n## Getting started\n\n```bash\ngo get github.com/MontFerret/ferret/v2@latest\n```\n\nThere are currently two ways to start with Ferret v2:\n- Native v2 API - recommended for new projects\n- `compat` module - recommended as a first migration step for existing v1 integrations\n\n### New projects\n\nUse the native v2 API built around the following flow:\n\n```\nEngine -> compile query -> create session -> run\n```\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com/MontFerret/ferret/v2/pkg/engine\"\n)\n\nfunc main() {\n\tctx := context.Background()\n\n\teng, err := engine.New()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer eng.Close()\n\n\tplan, err := eng.Compile(`return 1 + 1`)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tsession, err := plan.NewSession()\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\tdefer session.Close()\n\n\tresult, err := session.Run(ctx)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tfmt.Println(result.Content)\n}\n```\n\n### Migration from v1\n\nFerret v2 introduces a new architecture and public API, so existing Go applications should migrate in two stages: first to the v2 compatibility API, then incrementally to the native v2 API.\n\nRun the [`ferret migrate`](https://github.com/MontFerret/cli#migrating-embedded-ferret-applications) command from anywhere inside the application's Go module:\n\n```bash\nferret migrate --dry-run # List the files that would change\nferret migrate --print # Print a unified diff without changing files\nferret migrate # Apply the migration\n```\n\nThe command rewrites the documented v1 imports to their v2 compatibility packages, updates `go.mod` and `go.sum` as required by those rewrites, and formats changed Go files. Generated, vendored, and nested-module files are left untouched. Unsupported v1 imports are reported as manual follow-up; if the project vendors dependencies, run `go mod vendor` after applying the migration.\n\nThis is only the mechanical compatibility stage. The command does not convert application logic to the native v2 API or migrate drivers and other unsupported v1 packages. Running it again on an already-migrated compatibility project is a no-op and does not upgrade the Ferret v2 dependency merely because the CLI is newer.\n\nAfter applying the migration, address any reported manual follow-up, build and test the application, and then migrate to the native v2 API over time. The compatibility layer is a migration aid, not the long-term preferred API; new projects should use the native v2 packages directly.\n\n### Alpha status\n\nFerret v2 is currently in active development.\n\nAlpha releases are intended for early adopters, experimentation, and feedback. Some APIs and language features may still change before the stable v2 release.\n\n## Maintainers\n\n- [Versioned Ferret Core API Reference](docs/maintainers/core-api-reference.md)\n\n## Support Ferret\n\nFerret is supported by organizations and community members who help fund its continued development. [View all supporters](https://ferretlang.org/sponsor/).\n","default_branch":"main","files":1451,"tree":[".codecov.yml",".editorconfig",".gitattributes",".github/FUNDING.yml",".github/ISSUE_TEMPLATE/bug_report.md",".github/ISSUE_TEMPLATE/feature_request.md",".github/ISSUE_TEMPLATE/question.md",".github/dependabot.yml",".github/stale.yml",".github/workflows/benchmark.yml",".github/workflows/build.yml",".github/workflows/codeql.yml",".github/workflows/notify-dependents.yml",".github/workflows/publish-core-api.yml",".gitignore",".golangci.yml","AGENTS.md","CODE_OF_CONDUCT.md","CONTRIBUTING.md","LICENSE","Makefile","README.md","assets/intro.jpg","assets/logo.png","bootstrap.go","compat/compat.go","compat/compat_test.go","compat/compiler/compiler.go","compat/compiler/compiler_test.go","compat/options.go","compat/runtime/core/comparison_test.go","compat/runtime/core/core.go","compat/runtime/core/functions.go","compat/runtime/core/helpers.go","compat/runtime/core/scope.go","compat/runtime/runtime.go","compat/runtime/values/types/types.go","compat/runtime/values/types/types_test.go","compat/runtime/values/values.go","compat/runtime/values/values_test.go","debug_session_benchmark_test.go","debug_session_services.go","debug_session_test.go","debug_session_udf_resolution_test.go","debug_types.go","docs/maintainers/core-api-reference.md","engine.go","engine_helpers.go","engine_lifecycle_test.go","engine_options.go","engine_options_test.go","engine_test.go","engine_waitfor_without_stdlib_test.go","go.mod","go.sum","hooks.go","host.go","network_context_test.go","network_http_client_test.go","network_test_helpers_test.go","output.go","output_test.go","pkg/asm/assembler.go","pkg/asm/disassembler.go","pkg/asm/disassembler_options.go","pkg/asm/disassembler_test.go","pkg/asm/duration_test.go","pkg/asm/errors.go","pkg/asm/formatter.go","pkg/bytecode/aggregate_plan.go","pkg/bytecode/artifact/artifact.go","pkg/bytecode/artifact/artifact_bench_test.go","pkg/bytecode/artifact/artifact_test.go","pkg/bytecode/artifact/detect.go","pkg/bytecode/artifact/detect_test.go","pkg/bytecode/artifact/errors.go","pkg/bytecode/artifact/header.go","pkg/bytecode/artifact/loader.go","pkg/bytecode/artifact/marshal.go","pkg/bytecode/collector_type.go","pkg/bytecode/destructure_mode.go","pkg/bytecode/encoding.go","pkg/bytecode/errors.go","pkg/bytecode/format/format.go","pkg/bytecode/format/json/format.go","pkg/bytecode/format/json/format_test.go","pkg/bytecode/format/msgpack/format.go","pkg/bytecode/format/msgpack/format_test.go","pkg/bytecode/functions.go","pkg/bytecode/helpers.go","pkg/bytecode/host_function.go","pkg/bytecode/instruction.go","pkg/bytecode/internal/persist/frame.go","pkg/bytecode/internal/persist/frame_test.go","pkg/bytecode/limits.go","pkg/bytecode/opcode.go","pkg/bytecode/opcode_meta.go","pkg/bytecode/opcode_meta_test.go","pkg/bytecode/operand.go","pkg/bytecode/program.go","pkg/bytecode/program_json_test.go","pkg/bytecode/udf.go","pkg/bytecode/udf_display_name_test.go","pkg/bytecode/validation.go","pkg/bytecode/validation_test.go","pkg/compiler/analysis.go","pkg/compiler/analysis_build.go","pkg/compiler/analysis_helpers.go","pkg/compiler/analysis_test.go","pkg/compiler/compiler.go","pkg/compiler/debug_test.go","pkg/compiler/frontend_driver.go","pkg/compiler/internal/assignment_target.go","pkg/compiler/internal/ast_helpers.go","pkg/compiler/internal/ast_helpers_test.go","pkg/compiler/internal/binding_assignment_path.go","pkg/compiler/internal/binding_capture_info.go","pkg/compiler/internal/binding_compiler.go","pkg/compiler/internal/binding_compiler_test.go","pkg/compiler/internal/binding_delete.go","pkg/compiler/internal/binding_helpers.go","pkg/compiler/internal/binding_pattern.go","pkg/compiler/internal/call_resolver.go","pkg/compiler/internal/capture_analyzer.go","pkg/compiler/internal/concat_plan.go","pkg/compiler/internal/context.go","pkg/compiler/internal/core/aggregate_selector.go","pkg/compiler/internal/core/catch.go","pkg/compiler/internal/core/collect_selector.go","pkg/compiler/internal/core/collector.go","pkg/compiler/internal/core/collector_aggregation.go","pkg/compiler/internal/core/collector_projection.go","pkg/compiler/internal/core/constants.go","pkg/compiler/internal/core/constants_test.go","pkg/compiler/internal/core/emitter.go","pkg/compiler/internal/core/emitter_extension.go","pkg/compiler/internal/core/error_policy.go","pkg/compiler/internal/core/errors.go","pkg/compiler/internal/core/errors_test.go","pkg/compiler/internal/core/host.go","pkg/compiler/internal/core/host_function_table.go","pkg/compiler/internal/core/host_test.go","pkg/compiler/internal/core/invariant.go","pkg/compiler/internal/core/invariant_helpers.go","pkg/compiler/internal/core/invariant_test.go","pkg/compiler/internal/core/kv.go","pkg/compiler/internal/core/label.go","pkg/compiler/internal/core/loop.go","pkg/compiler/internal/core/loops.go","pkg/compiler/internal/core/recovery_plan.go","pkg/compiler/internal/core/registers.go","pkg/compiler/internal/core/retry.go","pkg/compiler/internal/core/scope.go","pkg/compiler/internal/core/scope_test.go","pkg/compiler/internal/core/symbols.go","pkg/compiler/internal/core/symbols_test.go","pkg/compiler/internal/core/type_tracker.go","pkg/compiler/internal/core/udf.go","pkg/compiler/internal/core/value_type_join.go","pkg/compiler/internal/core/value_type_join_test.go","pkg/compiler/internal/debug_point.go","pkg/compiler/internal/diagnostic_helpers.go","pkg/compiler/internal/dispatch.go","pkg/compiler/internal/expr.go","pkg/compiler/internal/expr_call.go","pkg/compiler/internal/expr_call_compiler.go","pkg/compiler/internal/expr_helpers_test.go","pkg/compiler/internal/expr_match.go","pkg/compiler/internal/expr_match_compiler.go","pkg/compiler/internal/expr_match_helpers.go","pkg/compiler/internal/expr_match_helpers_test.go","pkg/compiler/internal/expr_member.go","pkg/compiler/internal/expr_member_helpers.go","pkg/compiler/internal/expr_predicate.go","pkg/compiler/internal/expr_predicate_helpers.go","pkg/compiler/internal/expr_query.go","pkg/compiler/internal/expr_query_compiler.go","pkg/compiler/internal/expr_query_helpers.go","pkg/compiler/internal/forward_binding_index.go","pkg/compiler/internal/frontend.go","pkg/compiler/internal/literal.go","pkg/compiler/internal/literal_helpers.go","pkg/compiler/internal/loop.go","pkg/compiler/internal/loop_collect.go","pkg/compiler/internal/loop_collect_agg.go","pkg/compiler/internal/loop_collect_grp.go","pkg/compiler/internal/loop_collect_locals.go","pkg/compiler/internal/loop_collect_prj.go","pkg/compiler/internal/loop_helpers.go","pkg/compiler/internal/loop_sort.go","pkg/compiler/internal/match_fold_error_value_test.go","pkg/compiler/internal/name_collision_validator.go","pkg/compiler/internal/operand_normalization.go","pkg/compiler/internal/optimization/analysis.go","pkg/compiler/internal/optimization/analysis_test.go","pkg/compiler/internal/optimization/basic_block.go","pkg/compiler/internal/optimization/builder.go","pkg/compiler/internal/optimization/cfg.go","pkg/compiler/internal/optimization/cfg_test.go","pkg/compiler/internal/optimization/coalescing.go","pkg/compiler/internal/optimization/const_eval.go","pkg/compiler/internal/optimization/const_fold.go","pkg/compiler/internal/optimization/const_fold_result.go","pkg/compiler/internal/optimization/errors.go","pkg/compiler/internal/optimization/interference.go","pkg/compiler/internal/optimization/pass.go","pkg/compiler/internal/optimization/pass_constant_propagation.go","pkg/compiler/internal/optimization/pass_constant_propagation_test.go","pkg/compiler/internal/optimization/pass_liveness.go","pkg/compiler/internal/optimization/pass_liveness_test.go","pkg/compiler/internal/optimization/pass_peephole.go","pkg/compiler/internal/optimization/pass_peephole_test.go","pkg/compiler/internal/optimization/pass_register_coalescing.go","pkg/compiler/internal/optimization/pass_register_coalescing_test.go","pkg/compiler/internal/optimization/peephole_helpers.go","pkg/compiler/internal/optimization/peephole_remap.go","pkg/compiler/internal/optimization/pipeline.go","pkg/compiler/internal/optimization/pipeline_test.go","pkg/compiler/internal/optimization/pipeline_test_pass.go","pkg/compiler/internal/optimization/renumbering.go","pkg/compiler/internal/optimization/set_helpers.go","pkg/compiler/internal/optimization/use_def_collector.go","pkg/compiler/internal/parser_names.go","pkg/compiler/internal/policy.go","pkg/compiler/internal/recovery_compiler_plan.go","pkg/compiler/internal/recovery_compiler_retry.go","pkg/compiler/internal/recovery_helpers.go","pkg/compiler/internal/result_use.go","pkg/compiler/internal/semantic_recorder.go","pkg/compiler/internal/semantic_recorder_helpers.go","pkg/compiler/internal/stmt.go","pkg/compiler/internal/type_facts.go","pkg/compiler/internal/udf_capture_env.go","pkg/compiler/internal/udf_collect.go","pkg/compiler/internal/udf_compile.go","pkg/compiler/internal/udf_helpers.go","pkg/compiler/internal/wait_event.go","pkg/compiler/internal/wait_event_group.go","pkg/compiler/internal/wait_event_group_recovery.go","pkg/compiler/internal/wait_event_tail.go","pkg/compiler/internal/wait_helpers.go","pkg/compiler/internal/wait_polling.go","pkg/compiler/internal/wait_predicate.go","pkg/compiler/internal/wait_predicate_group.go","pkg/compiler/internal/wait_recovery.go","pkg/compiler/internal/wait_timeout.go","pkg/compiler/options.go","pkg/compiler/panic_recovery.go","pkg/compiler/program_build.go","pkg/compiler/semantic.go","pkg/compiler/syntax_tokens.go","pkg/compiler/syntax_tokens_test.go","pkg/compiler/syntax_words.go","pkg/compiler/visitor.go","pkg/debugger/blocking_execution_test.go","pkg/debugger/breakpoint_helpers.go","pkg/debugger/breakpoint_test.go","pkg/debugger/doc.go","pkg/debugger/eval.go","pkg/debugger/eval_arithmetic_test.go","pkg/debugger/eval_comparison_test.go","pkg/debugger/eval_listener.go","pkg/debugger/eval_scope.go","pkg/debugger/eval_test.go","pkg/debugger/fake_execution_test.go","pkg/debugger/fake_services_test.go","pkg/debugger/fake_value_access_test.go","pkg/debugger/format.go","pkg/debugger/format_bench_test.go","pkg/debugger/format_test.go","pkg/debugger/resume_context.go","pkg/debugger/resume_context_helpers.go","pkg/debugger/resume_context_test.go","pkg/debugger/session.go","pkg/debugger/session_close_test.go","pkg/debugger/session_concurrency_test.go","pkg/debugger/session_frame_resolution_test.go","pkg/debugger/session_test.go","pkg/debugger/state_error.go","pkg/debugger/types.go","pkg/diagnostics/diagnostic.go","pkg/diagnostics/diagnostics.go","pkg/diagnostics/diagnostics_test.go","pkg/diagnostics/error_span.go","pkg/diagnostics/error_span_test.go","pkg/diagnostics/formattable.go","pkg/diagnostics/formatter.go","pkg/diagnostics/formatter_test.go","pkg/diagnostics/helpers.go","pkg/diagnostics/kind.go","pkg/diagnostics/render.go","pkg/encoding/codec.go","pkg/encoding/context.go","pkg/encoding/errors.go","pkg/encoding/helpers.go","pkg/encoding/json/codec.go","pkg/encoding/json/codec_bench_test.go","pkg/encoding/json/codec_test.go","pkg/encoding/json/config.go","pkg/encoding/json/decoder.go","pkg/encoding/json/encoder.go","pkg/encoding/msgpack/codec.go","pkg/encoding/msgpack/codec_bench_test.go","pkg/encoding/msgpack/codec_test.go","pkg/encoding/msgpack/config.go","pkg/encoding/msgpack/decoder.go","pkg/encoding/msgpack/encoder.go","pkg/encoding/msgpack/helpers.go","pkg/encoding/output.go","pkg/encoding/registry.go","pkg/encoding/registry_test.go","pkg/formatter/case.go","pkg/formatter/formatter.go","pkg/formatter/formatter_parentheses_bench_test.go","pkg/formatter/formatter_parentheses_test.go","pkg/formatter/formatter_test.go","pkg/formatter/internal/binding_pattern.go","pkg/formatter/internal/case.go","pkg/formatter/internal/clause.go","pkg/formatter/internal/clause_test.go","pkg/formatter/internal/engine.go","pkg/formatter/internal/expression.go","pkg/formatter/internal/expression_parentheses.go","pkg/formatter/internal/expression_test.go","pkg/formatter/internal/keywords.go","pkg/formatter/internal/list.go","pkg/formatter/internal/list_test.go","pkg/formatter/internal/literal.go","pkg/formatter/internal/member.go","pkg/formatter/internal/options.go","pkg/formatter/internal/printer.go","pkg/formatter/internal/printer_test.go","pkg/formatter/internal/program.go","pkg/formatter/internal/statement.go","pkg/formatter/internal/statement_test.go","pkg/formatter/internal/strings.go","pkg/formatter/internal/test_helpers_test.go","pkg/formatter/internal/trivia.go","pkg/formatter/internal/trivia_test.go","pkg/formatter/internal/value.go","pkg/formatter/internal/visitor.go","pkg/formatter/options.go","pkg/fs/context.go","pkg/fs/context_test.go","pkg/fs/errors.go","pkg/fs/file.go","pkg/fs/fs.go","pkg/fs/fs_disabled.go","pkg/fs/fs_root.go","pkg/fs/fs_test.go","pkg/fs/helpers.go","pkg/fs/options.go","pkg/internal/debugpoint/index.go","pkg/internal/debugpoint/index_test.go","pkg/internal/operator/array_comparator.go","pkg/internal/operator/array_comparator_test.go","pkg/internal/operator/binary.go","pkg/internal/operator/binary_test.go","pkg/internal/operator/unary.go","pkg/internal/operator/unary_test.go","pkg/internal/valueset/set.go","pkg/internal/valueset/set_test.go","pkg/logging/level.go","pkg/logging/logger.go","pkg/logging/options.go","pkg/module/bootstrap.go","pkg/module/doc.go","pkg/module/hooks.go","pkg/module/host.go","pkg/module/module.go","pkg/net/context.go","pkg/net/context_test.go","pkg/net/errors.go","pkg/net/helpers.go","pkg/net/http/client.go","pkg/net/http/client_benchmark_test.go","pkg/net/http/client_constructor_test.go","pkg/net/http/client_hardening_test.go","pkg/net/http/client_security_internal_test.go","pkg/net/http/client_test.go","pkg/net/http/defaults.go","pkg/net/http/dns_resolver_test.go","pkg/net/http/doc.go","pkg/net/http/errors.go","pkg/net/http/header_validation_error.go","pkg/net/http/header_validation_test.go","pkg/net/http/headers.go","pkg/net/http/helpers.go","pkg/net/http/helpers_test.go","pkg/net/http/invalid_method_error.go","pkg/net/http/multi_policy_configuration_error.go","pkg/net/http/multi_policy_configuration_error_test.go","pkg/net/http/nil_response_guard_transport.go","pkg/net/http/options.go","pkg/net/http/policy.go","pkg/net/http/policy_configuration_error.go","pkg/net/http/policy_construction_test.go","pkg/net/http/policy_dialer.go","pkg/net/http/policy_enforcement_test.go","pkg/net/http/policy_error.go"],"storefront":"/r/MontFerret","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/MontFerret/ferret/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}