Releases

Features:

• Implemented permissions support for DEFINE API

• Introduced authentication rate limiting (AuthLimit)

• Introduced hash-based vector deduplication for HNSW indexes

• Exposed metrics for external storage composers

• Integrated CRUD benchmarking suite

Bug fixes:

• Fixed backwards compatibility issues in authentication limits

• Fixed HNSW layer popping logic

• Fixed GROUP validation and ORDER BY behavior when grouping

• Fixed edge cases in ONLY, OMIT, and SELECT VALUE

• Fixed edge cases in FETCH and ORDER BY

• Fixed ORDER BY behavior for sets and Unicode characters

• Fixed INSERT RELATION preserving extra fields when using table variants

• Fixed parameterized clause validation

• Fixed Linux Docker builder issues with OpenSSL and ONNX Runtime linking

• Fixed datastore shutdown failures

• Fixed supported SDK version reporting

• Removed unsupported VERSION clause from CREATE and INSERT

• Ensured api::invoke omits execution context where appropriate

Performance and optimisations:

• Improved GROUP and ORDER BY execution correctness

• Updated Nix dependencies

• Bumped surrealdb-protocol to v0.8.3

• Improved CI badge accuracy

• Added tests for SPLIT and SET support

• Added additional tests for START and LIMIT

• Added more language tests for ONLY and OMIT

• Added file-related tests

Features:

• Overhauled DEFINE API middleware logic

Bug fixes:

• Fixed race condition in Scanner causing duplicate range fetches

• Fixed panic in HNSW vector cache eviction under async contexts

• Fixed LIVE SELECT not returning UUID

• Fixed GraphQL parsing: excluded incorrectly treated as included

• Fixed bug in casting logic

• Fixed idiom continuity not broken by parentheses

• Fixed core contributor username in community labeler

• Fixed table serialization bug

• Fixed LIVE DIFF response structure

Performance and optimisations:

• Reduced instrumentation overhead in query record collectors

• Ensured explicit and prompt transaction closure to improve TiKV stability

• Ensured transactions explicitly finalized in IAM and RPC layers

• Improved edge lookups: Only fetch edge keys when values are unused

• Added criterion benchmarks for executor and functions

• Standardised storage engine metrics collection with initial RocksDB support

• Graph & recursive query testing

Features:

• Added Support for ORDER BY DESC on Indexes (RocksDB + TiKV)

• Added booleans as literal types

• Implemented TryFrom<RecordIdKey> for basic types

• Enable query planner on DELETE, UPDATE and UPSERT statements

• Added array functions for natural and/or lexical sorting

• Implemented reverse scan for table and range iterator

• Introduced Regex size limit configuration

• Implemented kind regex

• Added Filesystem support

• Added memory limit threshold check to all statements

• Added cbor encoding and decoding functions

• Imrpoved RPC protocol implementation

• Added support for distributed sequences

• Added SQL base64 encoding/decoding with & without padding

• Reduced storage and serialisation cost

• Added support for INFO FOR as a subquery

• Added support for filtering down graph selections with a range selection

• Optimised index iterators: stop scanning after LIMIT

• Exposed QueryType for live and kill responses

• Added RPC live statement options

• Added LQ Killed notification

• Allowed passing fields to RPC live

• Added support for partial array types in schemafull tables

• Introduced ALTER FIELD, update ALTER TABLE

• Ensured NS/DB is set; Ensure transaction is committed or cancelled

• Added Record type with metadata support

• Added support for multiple writers in Full-Text search implementation

• Enhanced Full-Text Search with OR Boolean Operation

• Added support for range scans on compound indexes (prefix + range)

• Added support for converting to/from protobufs and flatbuffers

• Introduced a None Type

• Added record::is::edge()

• Optimised START/LIMIT execution: correct semantics with WHERE/ORDER and early-stop iteration

• Added support for log files, json log format, and advanced log span filtering

• Introduced COUNT indexes for constant-time count() queries

• Added Infinity float value

• Added method syntax for record::is_edge()

• Exposed SurrealDB init as a library entrypoint with pluggable datastore builders

• Infered max_frame_size from max_message_size

• Introduced composable HTTP router factory for embedders

• Added string::capitalize function

• Added Support of standard deviation and variance in foreign tables

• Added environment variables for RocksDB BlobDB

• Use surrealdb_types::* in SDK and as core's public interface.

• Ensured reverse scan is used only with compatible datastore

• Introduced CommunityComposer and remove global config state

• Extended mock syntax to support full range

• Added array::sequence function

• Added configurable slow-query logging with parameter allow/deny filtering and normalized single-line output

• Added support for variables in closures and live queries

• Allow access to $input parameter in EVENTS

• Added support for multi-session support in RPC protocol

• Added Out of Disk (OOD) Error Handling for RocksDB Backend

• Implemented Distributed Batch-Based ID Generation for Catalog Entities

• Fixed CBOR and add type::of function.

• Added GraphQL back and upgrade to the latest async-graphql crates

• Added Value::Set / Set type

• Added refresh token support to the RPC protocol and SDK

• Added disk space limits and read-only/deletion-only modes for RocksDB

• Added support for Surrealism

• Added support for client-side transactions

• Added set literal syntax

• Improved DEFINE INDEX validation and diagnostics: enforce field count per index type with precise error spans

• Memory‑bounded LRU cache for HNSW vectors; remove deprecated MTREE index

• Stabilized record references & support reference by field

• Moved strict mode to DB definition, add strict object type

• Added improvements to views and $this, $parent

• Exposed cancellation token in datastore initialization API

• Use 32‑bit floats by default for HNSW vector indexes

• Added ALTER INDEX statement with PREPARE REMOVE for safe index decommissioning

• Added Default NS & DB

Bug fixes:

• Fixed array slice iterating over own items

• Fixed arithmatic wrap panic in duration functions

• Fixed index out of bounds error

• Fixed overflow and panics in math functions

• Improved ALL/ANYINSIDE behaviour with strings and arrays of strings

• Improved over-fetch documents when LIMIT and START used together

• Fixed Tokeniser fail with successive punctuations characters

• Fixed the default endpoint for exports and imports

• Fixed Graph clauses ->(SELECT ... issue in Rust SDK

• Fixed select with start and limit for arrays

• Fixed allowing null bytes in strings

• Fixed duplicate ID in GraphQL

• Fixed a bug with conversions to union types like array<int> | bool

• Fixed rand::time and other time-related issues

• Canonicalised allowed paths

• Normalised decimal values to fix indexing results

• Fixed concurrent indexing on quoted table names

• Fixed indexing on filtered count results

• Ensured arguments for object idiom methods are computed

• Fixed geometry parsing bug

• Reworked subqueries/covered expressions

• Allowed idiom functions to be restricted by capabilities

• Fixed regex_non_string.surql language-test for UUIDs

• Deprecate implicit LET

• Fixed memory usage and performance of Garbage Collection on old databases

• Fixed method calls on negative numbers

• Fixed JSON multipoint parsing

• Fixed START clause with WHERE; safer optimisation

• Fixed LIVE SELECT parsing with type::table()

• Limit fields in live query delete notifications

• Fixed invalid table view results with sum() returning 0

• Std index shouldn't fail if value already exists

• Fixed potential deadlock in distributed sequence allocation

• Introduced a NONE type

• Replaced BTreeDocIds with SeqDocIds in M-Tree, remove ANALYZE, use Strand for RecordKey::String

• Added edge name binding tests

• Enabled referencing fields in default value for defined fields.

• Added http connection pooling to maximise throughput.

• Removed dec when casting decimal to a string, allow string casting from None and Null

• Added idiom fallback function

• Added planner regression test for ORDER BY/LIMIT on compound index filters

• Restricted supported types on id field

• Included event errors in output

• Fixed incorrect ORDER BY LIMIT results on multi-column index scans

• Improved Parameterisation of statements

• Added notification for caller when a connection is dropped in Rust SDK

• Fixed warnings in language tests

• Fixed ignored insert with unique index creating a record when index violated

• Fixed strings, identifiers and escape sequences

• Fixed Windows file path handling in bucket operations

• Fixed WebSocket limits configurable and enforce max message size

• Fixed possible invalid formatting of idioms

• Fixed aliases not being respected in views

• Fixed invalid formatting of statement expressions within general expressions

• Ensured subqueries and blocks have consistent outputs

• Added CONTAINS/INSIDE planning consistent with substring semantics

• Ensured destructuring with computed fields returns correct output

• Improve behavior of group by and improve performance and robustness of views

• Fixed formatting bugs related to empty identifiers

• Removed Force Index mechanism and unify index building across all platforms

• fixed the info rpc

• Make Java SDK compatible with SurrealDB v3.0

• Fixed table serialization bug

• Fixed LIVE DIFF response data structure

• Ensured all statements return a value

• Fixed TableCheck output for TYPE RELATION / NORMAL

• Fixed SELECT with SPLIT so ORDER BY and START/LIMIT apply after splitting

• Fixed surrealism crates

• Added exact hint when path is deprecated

• Fixed Number hash to use consistent value hashing

• Fixed OOM crash in COUNT index on large datasets by batching key retrieval

• Fixed provided vars not being attached to context in surrealism 'sql_with_vars' function

• Fixed literal objects allow unknown keys

Performance and optimisations:

• Fetch ns, db, table, and fields once

• Reduced heap modifications in MemoryOrderedLimit

• Optimise index scans (descending) order and limit batches sizes

• Yield during CF/GC collection

• Unified lookups for graph & references

• Optimise some array methods

• Renamed Context and add trace level instrumentation.

• Invert expression value relation and move ast types out of value.

Storage/index format and protocol:

• Normalised numeric key encoding to unify ordering

• Introduced DecimalLexEncoder for Number

• Used IndexId for Index keys

• Stored keys using IDs for Namespace/Database

• Upgraded surrealdb-protocol to 0.4.0

• Improved key-value storage engine layer

Observability and developer experience:

• Implemented Slow logs

• Tokio Console integration

Security/networking:

• Reject IPs resolving to forbidden net targets

• DNS resolution on Context::check_allowed_net

• Configurable user agent for HTTP requests

Breaking changes:

• The rand::time function now returns a value between 0000-01-01T00:00:00Z and 9999-12-31T23:59:59Z instead of between 1970-01-01T00:00:00Z and +262142-12-31T23:59:59Z

• Removed legacy SEARCH full‑text index implementation; consolidate on FULLTEXT

• Dropped get() id special behaviour for complex record ids

• Changed Math functions to no longer return NONE

• Removed like operators

• Error for deprecated let-less let syntax

• Improved consistency in behaviour of functions

• Replaced <future> type with computed fields

• Renamed type::thing to type::record

• Renamed rand::guid() function to rand::id()

• Renamed additional type::thing to type::record

• Removed deprecated authentication statements

• Removed FoundationDB storage engine

• Format with backticks, deprecating angle brackets

• Added batched group commit functionality for RocksDB

Workflow improvements:

• Refactored depencies to use workspace dependencies

• Added more language tests and ports to the language suite

• Added /data and /logs directories to the Docker container

• Added Finnish to snowball filte

• Move yield_now to after work has been done

• Added additional clippy performance, soundness, and clarity lints

• Ensured a checksum is generated for Windows binaries

• Fixed JavaScript SDK compatibility tests

• Made sub-crates inherit workspace lints

• Ensured Clippy is applied to all sub-crates

• Added debugger setup instructions

• Removed Error::Ignore variant, moving into its own type

• Removed duplicate tagging job

• Improved test schema parsing and error handing

• Implement a sub-command to run upgrade tests with the language test suite

• Reused cargo workspace configuration

• Removed redundant key encoding & made Transaction dyn compatible

• Drop deprecated APIs in the SDK

• Remove unnecessary Killed notification from the API

• Moved errors over to use anyhow for future spliting of errors

• Split AST & Logical Plan into separate components

• Added cross-compile in CL as debug

• Fixed Python SDK tests.

• Fixed language tests floating point comparisons

• Pinned the rust toolchains for CI

• Improved authenticate configuration support in the testing suite

• Improved metrics consistency

• Add JSON serialization & deserialization tests

• TIKV client update to 0.3.0-surreal.3

• Removed cedar-policy dependency

• Overwrite by default during data import

• Prepare client-side transactions

• Default to FoundationDB 7.3

• Modified allocation tracking behaviour and implementation

• Improve context cancellation checking with adaptive back-off strategy for better OOM protection

• Improved Surrealism Wasm

Features:

• Implemented v3 compatible export

Bug fixes:

• Fixed HNSW layer pruning logic

• Updated in-memory caching to avoid storing entries with zero weights

Features:

• Added DEFER keyword for background index building and maintenance (Experimental)

• Added an auth_limit on defined apis, functions, fields and events, that limits execution to the permissions of the creating user instead of the invoking user.

Security Advisories:

• Confused Deputy Privilege Escalation through Future Fields and Functions

Note: Storable closures in future fields and functions are now disabled by default for security reasons. To enable the previous behavior, set the SURREAL_CAPS_ALLOW_INSECURE_STORABLE_CLOSURES environment variable. This change addresses the security vulnerability described in Advisory.

Features:

• Added CONTAINS/INSIDE planning consistent with substring semantics; use indexes only for flattened or set-based operators

• Made WebSocket limits configurable

• Added 32-bit floats by default for HNSW vector indexes

• Exposed and tune RocksDB BlobDB options via environment variables

• Aligned RocksDB blob defaults with upstream and make blob compression optional

• Added configurable slow-query logging with parameter allow/deny filtering and normalized single-line output

• Added disk space limits and read-only/deletion-only modes for RocksDB

• Added notification caller when a connection is dropped

• Added COUNT Index Support with Compaction System and Enable Concurrent Index Building for WASM

• Added minimal implementation for 2.0 → 3.0 migration tool

• Ensured reverse scan is used only with compatible datastore

Bug fixes:

• Fixed SELECT with SPLIT so ORDER BY and START/LIMIT apply after splitting

• Fixed OOM crash in COUNT index on large datasets by batching key retrieval

• Fixed insert ignore with unique index still creates a record when index violated

Breaking changes:

• CONTAINS/INSIDE planning made consistent with substring semantics - see language test example in this file with details on using the CONTAINSANY operator with indexes

Bug fixes:

• Fixed incorrect ORDER BY LIMIT results on multi-column index scans

• Fixed broken docs.rs

Bug fixes:

• Added HTTP connection pooling to maximize throughput

• Optimized START/LIMIT execution: correct semantics with WHERE/ORDER and early-stop iteration

• Optimized memory usuage by dropping mimalloc

Bug fixes:

• Fixed START clause with WHERE conditions and improve optimization safety

• Fixed allowing null bytes in strings

• Fixed Live Queries document issue

Improvements:

• Add configurable user agent to HTTP requests

• Support range scans on compound indexes (prefix + range)

• Yield during Change feed garbage collection

• Implemented slow logs for queries

• Added range scans support on compound indexes (prefix + range)

Security advisories:

• Unauthorized Data Exposure via LIVE Query Subscriptions

Bug fixes:

• Ensure session variables are attached to DEFINE API execution context

• The standard index should not fail if a value already exists

Bug fixes:

• Implements DNS resolution on Context::check_allowed_net

Workflow improvements:

• Added support for log files, json log format, and advanced log span filtering

• Improved metrics consistency

• Added support for Docker container timezone customisation

• Ensured errors are catched when checking permissions for LQ

• Ensured NS/DB is set; Ensure transaction is committed or cancelled

Security advisories:

• Bypass of deny-net flags via DNS resolution

Bug Fixes:

• TIKV client update to 0.3.0-surreal.3

Workflow improvements:

• Improved metrics consistency

• Reject resolution of ip addresses resolving to denied ip address when doing http request

Bug Fixes:

• Fixed memory usage and performance of Garbage Collection on old databases

Workflow improvements:

• Improved import memory usage

• Context cancellation check in prepare mock

• Changed default RocksDB configuration options

• Avoid using commit pool in wasm for skv in-memory

• Set default WS buffer size to 0, reactive RocksDB mapped R/W

Bug fixes:

• Fixed UPDATE permission to CREATE record

• Ensured caches of dependent resources are refreshed on TABLE OVERWRITE

Aligned arbitrary and experimental capabilities

This release resolves an issue identified in v2.3.0 that can corrupt the database when an UPDATE statement is stored within a function. If you experience this issue when migrating to v2.3.0 please kindly remove those function definitions, upgrade to v2.3.1 and then add them back.

Bug fixes:

• Fixed invalid revision on UPDATE statement

• Restored GraphQL, accidentally made inaccessible in the previous release

• Fixes invalid * value for SURREAL_CAPS_ALLOW_ARBITRARY_QUERY and SURREAL_CAPS_ALLOW_EXPERIMENTAL

NB: We identified an issue in this release that can corrupt the database when an UPDATE statement is stored within a function. We released v2.3.1 to address this. If you defined functions with UPDATE statements in v2.3.0 please kindly remove those function definitions, upgrade to v2.3.1 and then add them back.

Features:

• Added support for filtering down graph selections with a range selection

• Improved the parsing behaviour to accept any type of expression or start of idiom path

• Added Finnish to the snowball filter

• Implemented reverse scan for table and range iterator

• Ensured documents are not over-fetched when LIMIT and START are used together

• Added array functions for natural and/or lexical sorting

• Added /data and /logs directories to the Docker container, and set up Docker volumes for those directories by default

• Enabled the query planner on DELETE, UPDATE and UPSERT statements

• Implemented TryFrom<RecordIdKey> for basic types

• Added booleans as literal types

• Added support for ORDER BY DESC on indexes (RocksDB + TiKV)

• Implemented kind regex

• Improved ALL/ANYINSIDE behaviour with strings and arrays of strings

Bug fixes:

• Fixed rand::time and other time-related issues

• Fix the JSON response format

• Made GraphQL avoid duplicate ID fields

• Fixed sub-allocation in TrackAlloc

• Fixed the default endpoint for exports and imports

• Made several flow/timeout improvements

• Fixed tokeniser failure with successive punctuations characters

• Fixed overflow and panics in math functions

• Fixed possible index out of bounds on compound indexes

• Fixed arithmetic wrap panic in duration functions

• Fixed indexing on filtered count results

• Fixed concurrent indexing on quoted table names

• Fixed select with start and limit for arrays

• Enforced a limit to string::replace with regular expressions

• Ensured slicing an array does not iterate over its items

Deprecations:

The following Rust SDK APIs are now deprecated. In addition, some Capabilities methods were renamed and the previous names marked as deprecated.

• IntoQuery::into_query

• IntoEndpoint::into_endpoint

• IntoFn::into_fn

• IntoExportDestination::into_export_destination

• QueryResult::query_result

• QueryResult::stats

• QueryStream::query_stream

• IntoResource::into_resource

• CreateResource::into_resource

• proto::ResponseAction

• remote::WsNotification

Workflow improvements:

• Made the default Rust version explicit

• Allowed publishing multiple nightly crates per day

• Added Trivy scanner runs on new Docker images

• Added more language tests

• Refactored dependencies to use workspace dependencies

Bug fix:

• Fixed Live Queries document issue

Security advisories:

• Unauthorized Data Exposure via LIVE Query Subscriptions

Bug fixes:

• Ensure session variables are attached to DEFINE API execution context

• The standard index should not fail if a value already exists

Bug fixes:

• TIKV client update to 0.3.0-surreal.3

• Implements DNS resolution on Context::check_allowed_net

Workflow improvements:

• Added support for log files, json log format, and advanced log span filtering

• Improved metrics consistency

• Added support for Docker container timezone customisation

• Ensured errors are catched when checking permissions for LQ

• Ensured NS/DB is set; Ensure transaction is committed or cancelled

Security advisories:

• Bypass of deny-net flags via DNS resolution

Workflow improvements:

• Improved metrics consistency

• Reject resolution of ip addresses resolving to denied ip address when doing http request

Workflow improvements:

• Improved import memory usage

• Context cancellation check in prepare mock

• Changed default RocksDB configuration options

Bug fixes:

• Fixed escape issue on index specification

• [Fixed invalid "*" value for SURREAL_CAPS_[ALLOW_ARBITRARY_QUERY | ALLOW_EXPERIMENTAL]](https://github.com/surrealdb/surrealdb/pull/5890)

• Fixed select broken with index on count / where

Bug fixes:

• Check limit to string::replace with regular expressions

• Fixes long running FOR loops, adds script timeouts, improves HTTP redirect security, and optimizes task yielding for better performance

• Implements SURREAL_FILE_ALLOWLIST

• Improve error handling on value conversion (RPC / HTTP/SQL endpoint) (#5647)

• Introduces Regex size limit configuration (#5638)

• Backport Allocation tracking - 2.2.x (#5662)

• Add memory limit threshold check to all statements (#5704) (#5736)

• Add deal-store datasets to the upgrade test (#5738)

• Update branch name and pin Rust version for clippy and tests (#5768)

Security Advisories:

• Uncaught exception in Net module leads to database crash

• Server-takeover via SurrealQL injection on backup import

• Memory exhaustion via nested functions and scripts

• Local file read of 2-column TSV files via analyzers

• Memory exhaustion via string::replace using regex

• No JavaScript script function default timeout could facilitate DoS

• CPU exhaustion via custom functions result in total DoS

• Bypass of deny-net flags via redirect results in SSRF

Features:

• Allowed configuring experimental features in Rust SDK and CLI

• Introduced CLI Export customisation

Bug fixes:

• Ensured idiom fetching uses the LQ context

• Refactored capabilities to reduce allocations

• Ensured crates integrity

• Improved DEFINE API implementation

• Fixed DEFINE API stored key

• Improved array::fill

• Ensured NONE values are maintained in arrays

Features:

• Added shortest path queries

• Added support for graph clauses

• Added object::is_empty() function

• Added variable support for shortest path queries

• Added support of keys only logic with index iterators

• Added SURREAL_COUNT_BATCH_SIZE env variable

• Added type::is::range() method and idiom .is_range() method

• Added Support for memory threshold limits

• Added environment variables to diable tracing and metrics

• Added support for datetimes for rand::time function

• Added support of compound indexes on AND queries

• Enabled allocation tracking

• Added DEFAULT ALWAYS clause

• Implemented FETCH for live queries

Bug fixes:

• Fixed Index is corrupted - TreeNode out of bounds

• Enabled ANSI code support on Windows

• Fixed (-$param) failed bug in Parser

• Fixed index_hnsw bench

• Allowed escape characters inside mathematical angle bracket escaped idents

• Fixed allocator on Windows

• Fixed parse buffer not properly growing

• Reintroduced id selection behaviour

• Fixed design for live queries cache

• Fixed array::transpose() not adding NONE at non-existent indexes when transposing arrays of varying length

• Fixed concurrent index creation with overwrite

• Ensure INSERT IGNORE does not throw error

Optimizations:

• Introduced language testing suite

• Added cross-transaction caching to database and table statements

• Disallow mismatched field types

• Skip processing documents when START is specified

• Added minor optimisations for SELECT count() FROM table GROUP ALL queries

• Implemented priority queue for queries with ORDER and LIMIT

• Improved error messages for HTTP database functions

• Reworked query streaming and error reporting

• Disabled PARALLEL option

• Added file URL handling and tests for store initialization

• Added default limit for max open files with rocksdb storage engine

• Removed -log trace from Docker instructions

• Increased Rust Memory allocation, parallelism infos and RocksDB block cache size

• Added initial support for improved retryable transaction errors

• Simplified field processing logic

• Optimised transaction multi-get function interface

• Optimised RocksDB key-value gets

• Added stdout and stderr log splitting

• Added tests for other SDKs in CI

• Moved all experimental features to experimental capabilities

• Streamlined FoundationDB features

• Improved RocksDB block cache configuration

• Remove naive concurrency rate limiter in WebSocket

• Added dedicated threadpool for storage engine operations

• Skip strategy down to scanner for START queries

• Allow chaining path operations

• Implement support for RPC protocol versioning

• Switch to tokio::mpsc from async_channel in WebSocket

• Allow configurable background flush syncing

SurrealKV:

• Expose surrealkv config options as env variable

• Enable data durability configuration on RocksDB and SurrealKV

Workflow improvements:

• Fixed nightly version prefix

• Added Tx::count(), Tr::count(), and kvs::count() methods

• Fixed make check and cargo make ci-check-wasm commands

• Made all count/group use transaction::count method

• Documented core as internal only and remove doc(hidden)

• Added support for minimum number of tokio worker threads

• Removed development panic backtrace on transaction drop

• Fixed clippy warnings

• Added Compute args for idiom methods

• Ensured Windows builds have a default allocator

• Optimised Options and Auth core engine internals

• Added error logging for make sql and make repl

• Added Goodbye message to terminal before exiting

• Improved Issue templates

• Fixed GitHub issue labeling

Experimental:

• Changed bearer key storage and verification to only use hashes

• Implement refresh tokens for record users

• Added functions to GraphQL

• Added Record references to GraphQL

Breaking changes:

• array::transpose() now adds NONE at non-existent indexes when arrays vary in length.

Bug fixes and improvements:

• Added nu-ansi-term dependency

• Fixed Live Queries document issue

Security advisories:

• Unauthorized Data Exposure via LIVE Query Subscriptions

Bug fixes:

• TIKV client update to 0.3.0-surreal.3

• Implements DNS resolution on Context::check_allowed_net

Workflow improvements:

• Added support for log files, json log format, and advanced log span filtering

• Improved metrics consistency

• Added support for Docker container timezone customisation

• Ensured errors are catched when checking permissions for LQ

Security advisories:

• Bypass of deny-net flags via DNS resolution

Workflow improvements:

• Reject resolution of ip addresses resolving to denied ip address when doing http request

Workflow improvements:

• Improved import memory usage

• Context cancellation check in prepare mock

• Changed default RocksDB configuration options

Bug fixes:

• Check limit to string::replace with regular expressions (#5636)

• Fixes long running FOR loops, adds script timeouts, improves HTTP redirect security, and optimizes task yielding for better performance

• Fix import injection security bug

• Backport JS scripting mem exhaustion fix

• Implements SURREAL_FILE_ALLOWLIST

• Improve error handling on value conversion (RPC / HTTP/SQL endpoint) (#5647)

• Introduces Regex size limit configuration

• Backport Allocation tracking (#5663)

• Add is_beyond_threshold_check (#5667)

• Add deal-store datasets to the upgrade test

• Add memory limit threshold check to all statements (#5704) (#5737)

Security Advisories:

• Uncaught exception in Net module leads to database crash

• Server-takeover via SurrealQL injection on backup import

• Memory exhaustion via nested functions and scripts

• Local file read of 2-column TSV files via analyzers

• Memory exhaustion via string::replace using regex

• No JavaScript script function default timeout could facilitate DoS

• CPU exhaustion via custom functions result in total DoS

• Bypass of deny-net flags via redirect results in SSRF

Bug fixes:

• Support escape sequences inside idents escaped by mathematical angle brackets

Security advisories:

• Silent failure to overwrite table definition of relation type

Bug fixes:

• Added support for surrealkv+versioned in the binary

• Improved server termination handling, cleanup and responsiveness to pings

• Improved concurrency with long running queries

• Fixed parallel iteration support

• Ensured the Rust SDK can run queries against v2.0 servers

Bug fixes:

• Permission check should not generates an error when the table does not exist

• Document running in Docker as a non-root user

Bug fixes:

• Fixed behaviour on NONE and NULL targets

• Fixed live queries issue via the rpc method

• Simplified unnecessary versioned puts

• Check permissions on keys-only optimisation

SurrealKV:

• Disable versions by default

• Update SurrealKV to 0.6.1

Workflow improvements:

• Moved sub-crates into their own directories

• Ignore flaky transaction conflict tests

• Address Cargo Deny findings

• Fixed core and SDK paths

• Fixed benchmark workflow

Features:

• Recursive idiom/graph traversal

• Add cross-transaction caching to the document processing layer

• Added dictionary-based lemmatization to DEFINE ANALYZER statement

• More static values now possible after DEFAULT clause

• type::field() and type::fields() no longer restricted to just SELECT statements

• Now possible to configure what to export

• Allow data to be imported when server starts

• Added custom storage engine shutdown behaviour

• Versioned records now available in exports

• Enabled configuration of data in exports

• array::fold() and array::reduce() methods

• New string::distance and string::similarity functions for various algorithms

• Improvements to experimental bearer access

• New capabilities to limit HTTP and WebSocket APIs

• (Rust SDK) Allow constructing Datetime, better UUID support

• (SurrealKV) Versioning enabled on edges and relations

• Object Improvements

• Accept computation in VERSION clause

• SDK export config

• Add AND EXPUNGE clause to certain REMOVE statements

Performance improvements:

• Split collecting/processing + multithreading

• Remove execution-time parsing from RPC protocol

• SurrealKV now used for in-memory storage engine

• In-memory parallel sorting

• Efficient incremental ordering

• Added cross-transaction caching to document processing layer

• Query streaming for imports

• Improved performance when reduced document scopes not needed

Bug fixes:

• Fixed incorrect error message

• Fixed field not being removed when set to NONE

• Fixed panic caused by invalid UTF-8

• Fixed panic during parallel query calls in scripting functions

• Fixed bug in random ordering when using files as storage

• Fixed bytes serialization

• Fixed issue with lacking unicode escape sequences

• Fixed incorrect future CBOR decoding

• Fixed overly nested graph query output

• Fixed unexpected behaviour for optional operator

• Fixed OTEL endpoint

• Fixed ignoring of INDEX IGNORE

• Fixed panic when ordering by rand

• Fixed --auth-level flag not requiring necessary --username and --password

• Fixed like operator parsing error

• Fixed invalid exports containing array<any>

• Fixed string-like record ID parsing

• Gracefully handle Option<T> in rand::time()

• Allow customising record exporting

• Better handling of unexpected authentication errors

• Improve error messages for negative arithmetic + try_reserve

• Ensure LIMIT and PARALLEL can be used together

Workflow Improvements:

• Benchmark to compare indexed with non-indexed queries

• Improved error message when permissions insufficient

• RocksDB storage engine config options now shown on startup

• Tests to prevent regressions after permission fixes

• Use new runner types in benchmark jobs

• Update dependencies with known security vulnerabilities

• Update rquickjs to newest version

Features with breaking changes:

• Improved UPSERT behaviour

Bug Fixes:

• Check limit to string::replace with regular expressions

• Fixes long running FOR loops, adds script timeouts, improves HTTP redirect security, and optimizes task yielding for better performance

• Backport JS scripting mem exhaustion fix

• Introduces Regex size limit configuration

• Improve error handling on value conversion (RPC / HTTP/SQL endpoint)

• Update rquickjs to newest version

Security Advisories:

• Uncaught exception in Net module leads to database crash

• Server-takeover via SurrealQL injection on backup import

• Memory exhaustion via nested functions and scripts

• Memory exhaustion via string::replace using regex

• No JavaScript script function default timeout could facilitate DoS

• CPU exhaustion via custom functions result in total DoS

• Bypass of deny-net flags via redirect results in SSRF

Performance improvements:

• Added lower and upper bounds for RocksDB iterators

• Added with or without index benchmark

• Improved behaviour and performance of reduced document scopes

Bug fixes:

• Fixed panic when calling Thing::try_from("") - with empty value

• Fixed parsing of functions and models in selectors

• Fixed clippy warnings in release build

• Fixed micro second parsing

Workflow Improvements:

• Ensure unstable code is also checked in CI

• Update versions actively supported for security fixes

Security advisories:

• Improper Authorization in Select Permissions

• Uncaught Exception Handling Parsing Errors on Empty Strings

Features:

• Allowed transactions during imports

• Deprecated DELETE on field permissions

• Implemented Ord for Duration and Table

• Improved Number ordering accuracy

• Made max_value_size configurable in surrealkv

Parser Improvements:

• Update reblessive

• Fixed digit object name export

Bug fixes:

• Fixed record IDs not being used as Object keys

• Fixed ID type conversion

• Fixed thing parsing function

• Removed truthiness check on Datetime that requires after epoch dates

• Reintroduced meta:: scope as deprecated methods

• Fixed idiom path processing

• Fixed value convertion to a vector: NONE

Workflow Improvements:

• Update SurrealKV version to 0.3.7

• Update rocksdb 3rd party dependency

• Create major.minor and major tags when pushing to Docker

• Update surrealcs dependency

Bug fixes:

• Fixed fields being parsed as idioms.

• Fixed error message panics

• Fixed Incorrect lowercase filter in analyzer

• Fixed mismatch in record id for edge tables

• Fixed immediate insert subqueries

• Fixed single check for rpc insert

• Fixed decimal parsing

• Fixed failed authenticate resulting in session reset

• Fixed version patching

• Fixed query Planner Ignoring Isolated predicates in WHERE Clause

• Fixed inconsitent numeric object key

• Fixed ULID/UUID gen, programmatically generating ranges, RETURN inside FOR/IF and improves arithmetic operations

• Fixed CBOR Range encoding

• Fixed parsing bug in graph for bidirectional relationships

• Fixed compilation issue on 32bits platforms (other than wasm32)

• Fixed seting a range id on a document's id, in and out fields

• Fixed max key size for surrealkv store

• Included request id in CBOR encoding failure message

• Fixed panic when the query file size is too large.

• Fixed bug with number ordering

Improvements:

• Improved log messages for authentication errors

  • Explicitly check for token expiration error

• AddedGraphQL description exactly match the comments set by the user

• Added configuration capabilities for SurrealCS connection pool size

• Added test for including an ID in the content body

• Added minor improvements to core engine

• Added VERSION clause to all queries

• Added additional ULID and UUID utilities

• Added DTrace profiling support

Bug Fixes:

• Fixed infinite loop in parser

After numerous alpha and beta releases over months of development, we are releasing SurrealDB v2.0.0! 🎉. The changes for this release pertain to those in between the final 2.0.0-beta-3 and other alpha versions.

NOTE: Because of changes in the underlying way SurrealDB stores data, steps are required to migrate to 2.0.0 from previous versions. Please use the following steps for any existing databases:

• If you are moving from 1.x to 2.x, use the surreal fix command.

Features:

• Enable FoundationDB transaction timeout customization

• Implement missing modulo operator in new parser

• Restructure the parser around the concept of token gluing.

• Added DEFINE ACCESS statement to grant access to resources.

• Deprecated DEFINE SCOPE in favor of DEFINE ACCESS ... TYPE RECORD.

• Deprecated DEFINE TOKEN in favor of DEFINE ACCESS ... TYPE JWT.

• Deprecated DEFINE TOKEN ON SCOPE in favor of DEFINE ACCESS ... TYPE RECORD ... WITH JWT.

• Added customizable algorithm and key when issuing tokens for records users with the WITH ISSUER clause.

• Added customizable token and session duration to DEFINE USER and DEFINE ACCESS.

• Removed session expiration when not explicitly defined.

• Changed the INFO statement to redact secrets defined via DEFINE ACCESS.

• Changed HTTP request headers expected by SurrealDB to require the surreal- prefix.

• Removed the --auth flag in the CLI to enable it by default. Added --unauthenticated.

• Removed the --enable-auth-level flag in CLI to enable the behavior by default. Defaults to root.

• Changed authentication to expect level via the --auth-level flag or surreal-auth-* headers.

• Changed the default --bind address in the CLI from 0.0.0.0 to 127.0.0.1.

• Added the --no-identification-headers flag in the CLI to limit information leakage.

• Added type::range function.

• Added TEMPFILES clause to the SELECT statement.

• Introduce resource creation checks for DEFINE statement to avoid duplicates.

• Added string functions for IP string::is::ip,string::is::ipv4,string::is::ipv6.

• Added string functions for HTML: string::html::encode, string::html::sanitize.

• Added new math functions: math::acos, math::acot, math::asin, math::atan, math::clamp, math::cos, math::cot, math::deg2rad, math::lerp, math::lerpangle, math::ln, math::log, math::log2, math::log10, math::rad2deg, math::sign, math::sin, math::tan.

• Support batch import with INSERT statement. This also extends to relationships between tables using the INSERT RELATION statement

• Allow bridge connectivity in all docker images built

• Added define JWT access at the root level

• Allow authenticating with a token from the CLI

• Added Asynchronous building for indexes

• Improved HNSW persistence

• Added Range value support

• Added ability to SELECT with version, CREATE with version and INSERT with version on SurrealKV

• Added ENFORCED keyword to DEFINE TABLE

• SCHEMAFULL tables are now TYPE NORMAL by default

• Added literal types to the query language

• Added the .chain() method

• Added typed LET statements

• Added typed function returns

• Added record::exists() function

• Added new functions value::diff() and value:::patch()

• Array function changes and additions: array::all(), array::any(), array::filter_index(), array::find_index() functions can now take a value or a closure. Also added array::filter() and array::find() functions.

• Upgraded SurrealKV to v0.3

• More flexible create API for Rust SDK

• Query depth limits can now be configured

• Added GraphQL config

• Indexing operators can now take expressions

• Support for mixed number types on range queries

• Improved capabilities logic with specific allows able to override general denies

• "false" string no longer eagerly evaluated as false

• time::epoch constant now available

• Added geo::is::valid() function

• Method call syntax now works on integers

• Improved GraphQL record fetching

• Added AUTHENTICATE clause on RECORD access

• Enabled publishing SurrealDB ports in Docker for bridge networking connectivity

• Allow generating ULID and UUID based on a timestamp

• Allow unsetting namespace and database on RPC protocol

• Moved fetch resolution to a compute method

• Added ALTER TABLE statement

• Moved AUTHENTICATE clause to the root of DEFINE ACCESS statement to allow the clause to be used in every access method

• Added OVERWRITE clause for defined resources

• Added array::windows() function

• Added support for idiom destructuring in SurrealQL

• Added string::is::record function

• Added Method chaining for Database functions

• Added anonymous functions

• Added typed function returns

• Function return types now shown in INFO statement

• Type any can now act as an optional type

• Added time::is::leap_year() function

• Added environment variables to configure HTTP endpoint limits & JavaScript runtime limits

• Introduced savepoint capabilities

• Added experimental VERSION clause to INFO FOR DB and INFO FOR TABLE

• Added additional chained methods to number values

Performance improvements:

• SELECT/count optimization on table range

• Improvements to node membership processing and garbage collection

• Improvements to select count table scan

• Avoid potentially some methods twice

• Stabilise and use parser2 for parsing.

• KNN filtering with limit and KNN distance function.

Bug fixes:

• Fixed incorrectly parsed identifier characters

• Fixed record<T> casting from string

• Connection ID now stored inside session data

• Fixed certain identifier characters not being parsed correctly

• Fixed remote fields added to output result inside SELECT * queries

• Revise RecordAccess structure to aid v2 alpha upgrade

• Fixed GHSA-gh9f-6xm2-c4j2: Improper Authentication when Changing Databases as Scope User

• Fixed argument check for array::clump

• Fixed aborting task leading to an error

• Fixed non-enforced field permissions on WHERE statements

• Fixed failing search when search analyzer lacks tokenizers

• Fixed core dump when performing INSERT

• Fixed discarding without error when using RELATE statement on edge that already exists

• Fixed modification of arrays with JSON Patch semantics

• Fixed selecting directly from a graph edge failing to select all primary edges

• Fixes to issues with composite indexes

• Fixed INSERT not working in subqueries

• Fixed KNN brute force error when an index exists.

• Fixed Redefining HNSW index error.

• Check that a field exists before index creation.

• Fixed missing serialisation for math::inf.

• Fixed polygon serialization.

• Fixed panic and infinite loop in parser.

• Fixed URL parsing function when a port is present.

• Fixed display formats for access methods

• Fixed parser error to allow ns as an alias for NAMESPACE

• Added missing datetime prefix

• Removed duplicated badge for docker pulls count in README.md

• Fixed export for sql::filter::Filter type in surrealdb-core

• Updated deprecated version of Node.js

• Fixed ELSE IF pretty formatting

• Changed geometry point values to floats in CBOR

• Fixed authentication checks on root user websocket RPC session

• Ensured a transaction is always available when live queries are evaluated

• Reduced read lock duration while processing connection messages

• Ensured execution halts after the RETURN statement

• Ensured the server shuts down properly

• Fixed querying complex edge table

• Removed unnecessary backticks from INFO FOR statement output

• Fixed Live Query Panic issue

• Fixed validation on schemafull RELATE

• Fixed division by zero bug in math::mean() function

• Improved string::is::datetime and string::is::uuid

• Parse like (~) operator correctly

• Removed faulty flatten in graph traversal

• Fixed record<T> casting from string

• Removed ASSERT for type option<T> and value NONE

• Fixed aud field on a JWT claim decoding

• Removed unneeded return in matching

• Disabled default audience validation for JWKS

• Fixed errors in supply chain security workflow

• Enabled check for single and multiple token audiences

• Removed panic on invalid range scan

• Added table to reserved keywords

• Concurrent indexing no longer fails if record updated during build

• Unreachable code now returns an error

• DEFINE EVENT defaults to true when no WHEN clause specified

• Fixed bug in which NONE and NULL both < 1 and > 1

• Fixed inconsistency on queries using equality with or without index

• Create on rpc which returned an array now returns an object. Similar change made to Rust SDK

• Rust SDK fix retrieving multiple columns from the same response

• Improved consistency for surrealdb.query() and surrealdb.value() in JavaScript runtime

• Fixed ENFORCED keyword not displaying in INFO statement

• Fixed GHSA-64f8-pjgr-9wmr: Untrusted Query Object Evaluation in RPC API

• Improved math::mean() to support returning floats

• Improved namespace/database validation for GraphQL

• Ensured TIMEOUT clauses are processed correctly

• Fixed importing from 1.x

• Deprecated missing format inference on RPC protocol

Workflow improvements:

• Deprecated file://

• Added surreal fix command

• Exports no longer use explicit transactions

• /lib path renamed to /sdk

• Rename the next workflow to alpha

• Improved structured output for INFO statement

• Added database upgrade test from 1.5.x to 2.0.x

• Improved InfoStructure implementation on Ident to return the raw value

• Improved placeholder text when filling a bug report

• Seperate the supply chain security actions to its workflow

• Improve Cargo local testing

• Limit Cargo build jobs during tests with coverage

• Improved type safety in router API

• Improved hashing mechanism to aid MTree benchmarking

• Removed unnecessary cloning of DefineStatementAnalyzer

• Simplified string formatting

• Refactored transaction, caching, and key-value store interfaces

• Imported entry enum in src/sql/mod.rs

• Extracted duplicated code for storing interfaces to initialize_store

• Resolved a TODO: map_err to inspect error

• Improved pull request template

• Added Improvements to kvs module layer

• Updated rustc version to 1.80.0

• Enabled the ml feature in nightly and alpha releases

• Removed limit to Cargo jobs

• Restored bench-sdk-surrealkv

• Upgraded to http/hyper 10

• Upgrade upstream jsonwebtoken crate

• Bumped SurrealKV version to 0.3.1

• Added aliases for complex feature flag combinations

• Improvemed RocksDB range scans

• Improved Serialisation for sql::Value

• Improved parser error type

• Added Support for SurrealCS

• Bumped async graphql version

• Improved transparent API

• Fixed cargo dependency loading issue

• Improved the logic for migrating IDs in surreal fix

API improvements:

• Decoupled capabilities from the public API

Surrealkv API improvements:

• Changed disk format for compaction in Surrealkv

• Removed global static Datastore

• Added versioning to SELECT * FROM table queries

Parser improvements:

• Added support for the s string prefix in parser

• Fixed parser panic when a dot follows a string in idiom-allowed contexts

• Ensured variables after the FETCH clause are parsed

• Ensured UUIDs are parsed correctly

• Fixed floating point parsing bug

• Updates reblessive dependency to newest version

• Removed flume as a dependency in favour of async-channels

• Fixed warnings and simplified connection trait

Breaking changes:

• Functions beginning with meta:: renamed to record::

• Functions string::endsWith() and string::startsWith() renamed to string::ends_with() and string::starts_with()

• Changed UPDATE statement behaviour of creating records in favour of new UPSERT statement.

Experimental features:

• Added BEARER access type and its basic grant management

Features:

• Created v1, v1-dev, v1.5 and v1.5-dev Docker tags

Bug fixes:

• Fixed an issue with object keys starting with a digit not being properly exported

Bug fixes:

• Fixed GHSA-64f8-pjgr-9wmr: Untrusted Query Object Evaluation in RPC API

Bug fixes:

• Allowed fields production to end a subquery

• Fixed GHSA-gh9f-6xm2-c4j2: Improper Authentication when Changing Databases as Scope User

Features:

• Added an --alpha flag to surreal upgrade.

Bug fixes:

• Fixed KNN brute force error when an index exists.

• Fixed redefining HNSW index error.

Bug fixes:

• Fixed incorrect computations on aggregations within foreign tables

• Fixed surreal upgrade --nightly

Bug fixes:

• Make temporary table active only if the temporary directory is set

• Fixes compilation issue related to temporary-directory when the feature sql2 is not enabled

Features:

• Added an implementation of HNSW in memory.

• Added a REBUILD INDEX statement.

• Added variable support in CONTENT clause for RELATE statements.

• Added more information to INFO FOR SCOPE.

• Added a relate method to the RPC protocol.

• Added an INFO STRUCTURE statement.

• Stabilised sql2 and jwks features.

• Introduced an on-disk temporary table.

• Added a run method to the RPC protocol.

• Implemented limits for parsing depth in the new parser.

• Implemented reblessive into the parser to prevent any overflows.

• Increased the Minimum Supported Rust Version (MSRV) to 1.77.

Bug fixes:

• Made the query planner recognise the exact operator (==).

• Fixed math:min in foreign tables.

• Fixed typo in function math:sum: was math::sun.

• Made record IDs more flexible in the new parser.

• Reverted changefeed polling frequency back to 10 seconds.

• Fixed a problem with date-time parsing.

• Fixed response content-type on /rpc endpoint.

• Fixed decimal decoding.

• Made the query planner support the IN operator.

• Made the JWKS implementation more RFC 7517 compliant.

• Fixed wrong count when using COUNT with a subquery.

• Fixed the content type header on a CBOR HTTP response.

• Fixed $value being NONE for DELETE events.

• Fixed array::group in a group by query.

Performance improvements:

• Added a query planner strategy for record links.

• Made TreeCache more efficient on writes.

• Reduced byte size of datetime and UUID types using CBOR format.

Bug fixes:

• Fix problems with if in identifiers after remove and define statements.

• Fix $value being NONE for DELETE events.

• Fix CBOR headers

• Fix wrong count when using COUNT with a subquery

• Fix IN operator should be recognised by the query planner

• Fix response content-type on /rpc endpoint

When using the Rust SDK, make sure surrealdb-core is up-to-date. If not, you might get back Revision errors as a result.

Features:

• Added a warning message about debug builds in CLI start.

• Moved JWKS cache storage to memory.

• Overhauled force implementation.

• Added DEFINE TABLE ... RELATION.

• Allowed RPC calls to be made over an HTTP connection, not just WebSocket.

• Allowed the highlighter to only highlight the matching searched token rather than the whole term.

• Added an INCLUDE ORIGINAL option to change-feeds.

• Added an insert method to the Rust SDK to allow bulk inserts.

Bug fixes:

• Fixed export generating unparsable code with the new parser.

• Eliminated a potential panic in MsgPack format implementation.

• Fixed string::is::longitude RegEx.

• Improved CBOR decoding.

• Fixed relation type parsing.

• Fixed handling of empty array on index.

• Allowed legacy headers in CORS.

• Allowed surreal upgrade to detect when upgrading to the same version and return early.

• Fixed certain environment variables to allow configuration at runtime.

• Reduced the byte size of datetime and uuid types using CBOR format.

• Fixed array::group in a group by query.

Performance improvements:

• Improved query aggregation handling.

  Bulk insert support in the Rust SDK

This was one of the frequently requested features. While this was already possible using the query method, this release adds an insert method that makes it more convenient.

db.insert(Resource::from("person"))
    .content(vec![
        User {
            id: thing("person:tobie")?,
            name: "Tobie",
            settings: Settings {
                active: true,
                marketing: false,
            },
        },
        User {
            id: thing("person:jaime")?,
            name: "Jaime",
            settings: Settings {
                active: true,
                marketing: true,
            },
        },
    ])
    .await?;

Bug fixes:

• Add JWT ID claim to tokens issued by SurrealDB.

• Consistently enforce session expiration.

Features:

• Introduced IF NOT EXISTS clause on DEFINE statements.

• Implemented IF EXISTS for additional REMOVE statements.

• Changed the KNN operator from <2> to <|2|> in the new parser and added support in the old parser for the new syntax.

• Implemented WebSocket session expiration.

• Added support for tables and geometries in CBOR.

• Added support for parsing decimal numbers with scientific notion.

• Added support for printing a warning in the CLI when using an outdated version of SurrealDB.

• Added Surreal::wait_for to make it possible for the client to wait for certain events.

• Added SurrealKV as an unstable feature.

• Added more error message improvements for the new parser.

Bug fixes:

• More consistent handling of negative numbers in record IDs.

• Removed the unimplemented backup command from the CLI.

• Fixed document not being available to delete permissions clause.

• Ensured objects properties are recognized by the query planner.

• Implemented the union strategy on unique indexes.

Performance improvements:

• Added compile-time feature for flamegraph and pprof profiling.

  IF NOT EXISTS clause on DEFINE statements

DEFINE statements now allow you to include an IF NOT EXISTS clause which ensures that the define statement is only run if the resource doesn't already exist. If it exists, you will get an error. Learn more about this in the documentation

DEFINE DATABASE IF NOT EXISTS example

New KNN syntax

The KNN operator now supports a new syntax using <| and |> instead of < and >. Learn more about this in the documentation

SELECT id FROM point WHERE point_b <|2|> [2, 3, 4, 5]

Waiting for client events

It's now possible to initialise the connection and run setup actions like authentication and selecting the database concurrently with other queries by making it possible to wait for the SDK to connect or select the database to use before allowing other queries to execute.

Something like this would be susceptible to a race condition before:

// A static, global instance of the client
static DB: Lazy<Surreal<Any>> = Lazy::new(|| {
    // Connect and setup the database connection in the background
    tokio::spawn(async {
        // This example is for demonstration purposes only.
        // If the `unwrap` panics the client will be left hanging,
        // waiting for the setup to happen when using `Surreal::wait_for`.
        setup_db().await.unwrap();
    });
    // Initialise a new unconnected instance of the client
    Surreal::init()
});

// Connect to the server and run setup actions
async fn setup_db() -> surrealdb::Result<()> {
    // Connect to the server
    DB.connect("ws://localhost:8000").await?;

    // Signin as a namespace, database, or root user
    DB.signin(Root {
        username: "root",
        password: "root",
    }).await?;

    // Select a specific namespace / database
    DB.use_ns("namespace").use_db("database").await?;

    Ok(())
}

// Whether this would run successfully or return an error would depend on whether
// `DB::setup` was able to execute before getting to this point.
DB.create(Resource::from("person")).await?;

Now it's possible to make the client wait for either the connection to be established

use surrealdb::opt::WaitFor::Connection;

// This will wait until the connection is established.
// As soon as it is, it will return and allow the client to continue.
DB.wait_for(Connection).await;

// This will still likely return an error.
// While we know that the connection is already established at this point,
// there is no guarantee that the database has already been selected.
// This query requires the database to be selected first.
DB.create(Resource::from("person")).await?;

or the database to be selected.

use surrealdb::opt::WaitFor::Database;

// This will wait until the database is selected.
// As soon as it is, it will return and allow the client to continue.
DB.wait_for(Database).await;

// At this point, the database is guaranteed to be selected already
// so this query should run successfully.
DB.create(Resource::from("person")).await?;

Bug fixes:

• Ensure relevant indexes are triggered when using IN in a SELECT query.

• Ensure the query planner accepts Datetime and Uuid values.

• Restore cosine distance on MTree indexes.

• Fix regression in index data serialisation.

• Ensure rquickjs builds don't stall on macOS.

Bug fixes:

• Fix an issue with WHERE clause on queries resolving record links or graph edges.

• Fix MATH::SQRT_2 not parsing.

• Fix a panic in span rendering.

• Fix CLI output not displaying properly sometimes.

Features:

• Bump MSRV to 1.75.

• In-memory index store.

• Show execution time in CLI.

• knn brute force.

• Implement support for remote JSON Web Key Sets.

• Add support for LIVE SELECT in the SDK and CLI.

• Add IF EXISTS to REMOVE TABLE statement.

• Add READONLY keyword to DEFINE FIELD statement.

• Add alias -V and --version flags for surreal version command.

• Define types for subfields when defining array fields.

• Add string::semver::compare, string::semver::major, string::semver::minor, string::semver::patch, string::semver::inc::major, string::semver::inc::minor, string::semver::inc::patch, string::semver::set::major, string::semver::set::minor, string::semver::set::patch methods.

Bug fixes:

• Make record id string parsing never fail early.

• Prevent overflow in math::power().

• Fix error message pointing to wrong character.

• Respect alias for dynamic field queries with type::field.

• Remove min/1000 in tx:delp.

• Replace custom JWT parser causing decoding issues.

• Issue with scoring on complex queries.

• Limit recursion depth when parsing nested RELATE statements.

• Fix a bug where a non-empty list parser would parse empty lists.

• Ensure an attempt to set a protected variable returns an error.

• Fix duration addition in timeout causing overflow.

• Panic invoking parameters and functions without a database.

• Fix a bug where the kind parser would eat whitespace.

• Fix WebSocket notification sending format.

• Implement missing errors for missing clauses on DEFINE-statements.

• Ensure advanced DEFINE PARAM parameters are computed correctly.

• Ensure path idioms are correct when looping over.

• Fix json failing to parse with trailing whitespace.

• Fix the five second delay in Wasm after initial connection.

• Add context to live query notifications.

• Fix the modulo operator on sql2 and parser1.

• Improve the js-surql value conversion for numbers.

• Implement revision types for client/server communication.

• Fix builtin error pointing to the wrong part.

• Fix a panic when invalid builtin function names are passed to the executor.

Performance improvements:

• Ensure compression is only enabled when response is a certain size.

• In MTree large vector improvement.

  Use JWKS to dynamically configure your token definitions

-- Specify the namespace and database for the token
USE NS abcum DB app_vitalsense;

-- Set the name of the token
DEFINE TOKEN token_name
  -- Use this token provider for database authorization
  ON DATABASE
  -- Specify the JWKS specification used to verify the token
  TYPE JWKS
  -- Specify the URL where the JWKS object can be found
  VALUE "https://example.com/.well-known/jwks.json"
;

Define a READONLY field

DEFINE FIELD created ON resource VALUE time::now() READONLY;

Define types for subfields when defining array fields

DEFINE FIELD foo ON bar TYPE array<number>
-- Where <number> is a subfield of the array type.
-- It now defines the number subfield automatically instead of having to run this manually:
DEFINE FIELD foo.* ON bar TYPE number

Bug fixes:

• Fix WebSocket notification sending format.

• Fix missing custom claims from token parameter.

• Fix URL encoding in JS functions.

• Fix panic when invoking parameters and functions without a database.

Features:

• The type::is::record() function now accepts a second optional table argument, validating the record being - stored on the passed table.

• Add time::micros(), time::millis() and time::from::nanos functions.

• Add type::is::none() function.

• Add object::entries(), object::from_entries(), object::len(), object::keys- () and object::values() functions.

• Clean paths in the start command and honour ~.

• CLI: Split results by comment.

• Add surreal sql welcome message.

• Add Debugging env flag: SURREAL_INSECURE_FORWARD_SCOPE_ERRORS.

• Add SURREAL_ROCKSDB_KEEP_LOG_FILE_NUM environment variable (default 20).

• Support auth levels for basic auth (behind feature flag)

• Add remainder/modulo operator.

• Implement string prefixes: s, r, d and u.

• Add ability to cast string to a Thing/Record ID.

• Analyzers to support functions.

• Support of subfields for embedding indexing.

• Add live query API to Rust SDK.

• Add Query::with_stats() to return query statistics along with the results.

• Permissions are now always displayed for visiblity

• Add a --beta flag to surreal upgrade to make installing the latest beta release easier.

Bug fixes:

• Fix stack overflow in graph traversal.

• Bugfix - parse error for invalid leading whitespace.

• Fix memory leak caused by OTEL callbacks.

• Fix wrong function name export and function name parsing.

• The position of the LIMIT and ORDER clauses are now interchangable.

• Fix index plan for idiom param value.

• Fix bug where error offset could underflow.

• Query results should be consistent and representative.

• Indexes used with the operators CONTAINS [ ANY | ALL ].

• Forward custom thrown errors in SIGNIN and SIGNUP queries.

• Fix ORDER BY RAND() failing to parse when selecting specific fields.

• Fix identifiers which look like numbers failing to parse.

• Change math::median indexing for even length arrays.

• Pass IP & Origin onto session used by scope queries.

• Fix possible corruption of MTree and incomplete knn.

• Allow array::flatten() to be used as an aggregate function.

• Make SELECT ONLY deterministic.

• Optional function arguments should be optional.

• Default table permissions should be NONE

• Bugfix: Fix inconsistant record parsing

• Fix time regression in surrealdb.wasm binaries.

• Fix computing futures in query conditions.

• Fix issue with scoring on complex queries.

• Fix ML support on Windows and enable the feature in Windows binaries.

• Replace the custom JWT parser causing decoding issues.

• Ensure compression is only enabled when response is a certain size.

• Respect alias for dynamic field queries with type::field().

• Prevent overflow in math::power().

• Fix error message pointing to wrong character.

• Expand logic for static value validation to improve DEFAULT clause handling.

• Fallback to a string when record ID parsing fails.

• Ensure an attempt to set a protected variable returns an error.

• Fix duration addition in timeout causing overflow.

• Fix a bug where a non-empty list parser would parse empty lists.

• Limit recursion depth when parsing nested RELATE statements.

• Ensure REMOVE statement does not only remove the first 1000 keys.

• Fix BTree deletion bug.

• Replace close method on live::Stream with a Drop trait implementation.

Performance improvements:

• Enable compression on the HTTP connector.

• Make REMOVE [ TABLE | DATABASE | NAMESPACE ] faster for TiKV and FoundationDB.

• Repetitive expressions and idioms are not anymore re-evaluated.

• Improve performance of CREATE statements, and record insertion.

• Improve RocksDB performance and configuration, introducing SURREAL_ROCKSDB_THREAD_COUNT, SURREAL_ROCKSDB_WRITE_BUFFER_SIZE, SURREAL_ROCKSDB_TARGET_FILE_SIZE_BASE, SURREAL_ROCKSDB_MAX_WRITE_BUFFER_NUMBER, SURREAL_ROCKSDB_MIN_WRITE_BUFFER_NUMBER_TO_MERGE, SURREAL_ROCKSDB_ENABLE_PIPELINED_WRITES, SURREAL_ROCKSDB_ENABLE_BLOB_FILES, SURREAL_ROCKSDB_MIN_BLOB_SIZE environment variables.

• Improve SpeeDB performance and configuration, introducing SURREAL_SPEEDB_THREAD_COUNT, SURREAL_SPEEDB_WRITE_BUFFER_SIZE, SURREAL_SPEEDB_TARGET_FILE_SIZE_BASE, SURREAL_SPEEDB_MAX_WRITE_BUFFER_NUMBER, SURREAL_SPEEDB_MIN_WRITE_BUFFER_NUMBER_TO_MERGE, SURREAL_SPEEDB_ENABLE_PIPELINED_WRITES, SURREAL_SPEEDB_ENABLE_BLOB_FILES, SURREAL_SPEEDB_MIN_BLOB_SIZE environment variables.

• Improve WebSocket performance, introduce SURREAL_WEBSOCKET_MAX_FRAME_SIZE, SURREAL_WEBSOCKET_MAX_MESSAGE_SIZE, SURREAL_WEBSOCKET_MAX_CONCURRENT_REQUESTS environment variables.

• Use specific memory allocators depending on OS.

• Fix memory leak in Websocket implementation.

  Get realtime updates in your Rust application with the Live Query API

v1.1.0 introduces a new Live Query API to the Rust SDK, for powerful realtime updates in your Rust applications.

// Select the namespace/database to use
db.use_ns("namespace").use_db("database").await?;

// Listen to all updates on a table
let mut stream = db.select("person").live().await?;

// Listen to updates on a range of records
let mut stream = db.select("person").range("jane".."john").live().await?;

// Listen to updates on a specific record
let mut stream = db.select(("person", "h5wxrf2ewk8xjxosxtyc")).live().await?;

// The returned stream implements `futures::Stream` so we can
// use it with `futures::StreamExt`, for example.
while let Some(result) = stream.next().await {
  handle(result);
}

// Handle the result of the live query notification
fn handle(result: Result<Notification<Person>>) {
  match result {
    Ok(notification) => println!("{notification:?}"),
    Err(error) => eprintln!("{error}"),
  }
}

Object functions

It was previously impossible to iterate over objects, so we introduced some new functions to make working with object data structures easier.

LET $fruits = {
  apple: {
    name: "Apple",
    stock: 20,
  },
  banana: {
    name: "Banana",
    stock: 40,
  },
};

LET $num_fruit_types = object::len($fruits);
LET $num_fruit_total = math::sum((
  SELECT VALUE stock FROM object::values($fruits)
));

RETURN "We have " + <string> $num_fruit_types + " type of fruits.";
RETURN "We have " + <string> $num_fruit_total + " pieces of fruit in total.";

String prefixes

Strings can optimistically be parsed as Record IDs, Datetimes or as a UUID, if the content matches such a value. With string prefixes you get to decide what value a string holds.

-- Interpeted as a record ID, because of the structure with the semicolon:
RETURN "5:20";
-- Forcefully parsed as just a string
RETURN s"5:20";

-- This will be a record ID.
RETURN r"person:john";
-- This fails, as it's not a valid record ID.
RETURN r"I am not a record ID";

-- Example for a date and a UUID.
RETURN d"2023-11-28T11:41:20.262Z";
RETURN u"8c54161f-d4fe-4a74-9409-ed1e137040c1";

Deterministic SELECT ONLY

The ONLY clause is sometimes not deterministic. Selecting from an array, table or range with the ONLY clause now requires you to limit the result to 1.

-- Fails, not limited to 1 result and result can contain multiple outputs.
SELECT * FROM ONLY table_name;

-- Works! Resource gets returned, or NONE if no resource was found.
SELECT * FROM ONLY table_name LIMIT 1;

Optional function arguments

Optional function arguments on custom functions are now actually optional.

DEFINE FUNCTION fn::create::resource($required: string, $optional: option<string>) {
  // The $required argument is a string
  // The $optional argument is a string or NONE
};

// Previously you needed to pass NONE for optional arguments
fn::create::resource("Required argument", NONE);

// Now you simply omit the argument
fn::create::resource("Required argument");

Bug fixes:

• Support connecting to beta servers from the Rust SDK.

Bug fixes:

• Add a patch for GHSA-x5fr-7hhj-34j3.

• Tables defined without explicit permissions have NONE instead of FULL permissions.

• Table permissions are always explicitly displayed with the INFO FOR DB statement.

After numerous beta releases crammed into just months of development, we are releasing SurrealDB v1.0.0! 🎉

Here follow some of our 1.0.0 highlights

Introducing new type validation methods in SurrealQL

v1.0.0 introduces new type validation methods. These new methods allow you to check which type any sort of value is on the go.

LET $value = "I am a string";

RETURN [
  type::is::string($value),
  type::is::record($value),
  type::is::number($value),
];

Guaranteed single item results with the new ONLY keyword

It was a previously difficult to select, create, update or delete just a single record, so we simplified it a bit.

// Will return an object, as we can guarantee just a single record is requested
SELECT * FROM ONLY person:tobie;

// This will throw an error, as multiple records are being created and returned
CREATE ONLY person:tobie, person:jaime;

Get realtime updates in your application with SurrealDB Live Queries

v1.0.0 introduces Live Queries. This powerful technology allows you to write applications where you can serve realtime updates to your frontend.

// You can initiate the live query in your application
LET $lq = LIVE SELECT * FROM person WHERE age > 18;
// And you can dispose the live query once it is no longer needed
KILL $lq;

// SurrealDB SDKs allow you to easily initiate live queries, and to process the received messages.
const lq = await db.live('person', function ({ action, result }) {
    if (action == 'CREATE') {
        renderPersonInFrontend(result);
    }
});

// Once you no longer need the live feed of updates, you can dispose the live query
await db.kill(lq);

Efficiently index and search your data with SurrealDB Full Text Search

With Full Text Search, you can efficiently store and index data, and search through it.

// We can define an analyzer on our database,
// Then, define an index to dictate which content we want to index (yep, that easy)
DEFINE ANALYZER simple TOKENIZERS blank,class FILTERS snowball(english);
DEFINE INDEX content ON article FIELDS content SEARCH ANALYZER simple BM25 HIGHLIGHTS;

// We can then create some content
CREATE article SET content = "Join us at SurrealDB World, as we unveil our version 1.0.0 to the world!";
CREATE article SET content = "We will absolutely be at Surreal World!";

// And lastly, select some data from the article table.
// We use the special "@num@" operator, where we define a reference.
// We can then reference back in the search::* functions.
SELECT
  *,
  search::score(1) AS score,
  search::offsets(1) AS offsets,
  search::highlight('<b>', '</b>', 1) AS highlighted
FROM article WHERE
  content @1@ 'world'
ORDER BY score DESC;

Allow or deny capabilities when starting your SurrealDB instance

All capabilities are disabled by default. This means that by default, you are not able to use any methods, embedded scripting functions, make outbound network calls, or access the database anonymously. Down below follows a set of examples to showcase how one can configure capabilities.

Capabilities are further documented in the Capabilities documentation.

# Allow all capabilities
user@localhost % surreal start --allow-all

# Allow all functions, except for custom functions
user@localhost % surreal start --allow-funcs --deny-funcs fn

# Allow all capabilities, but deny guest/anonymous access to the database
user@localhost % surreal start --allow-all --deny-guests

Revamped root users

It is now possible to define multiple root users in SurrealDB. This change did require some changes in the way that you start your database however.

With this change, you will now only initially have to provide the --user and --pass flags to create the initial root user, but once the first root user exists, they will no longer by utilized.

For more information, check out the Authentication guide, and the surreal start and DEFINE USER documentation.

# When you initially start the database, you can create the first root user.
user@localhost % surreal start --auth --user root --pass root file:database.db

# In the future, you don't need to specify the --user and --pass flags anymore
user@localhost % surreal start --auth file:database.db

// Afterwards, you are able to create multiple root users.
DEFINE USER tobie ON ROOT PASSWORD "SecurePassword!" ROLES OWNER;
DEFINE USER jaime ON ROOT PASSWORD "SecurePassword!" ROLES EDITOR;
DEFINE USER john  ON ROOT PASSWORD "SecurePassword!" ROLES VIEWER;