Schema-driven fuzzing
The ExploresOpenApiEndpoint trait ships for Laravel only. It generates deterministic request inputs for one operation or a filtered whole spec. Symfony, Pest, and plain PHPUnit users call the framework-agnostic OpenApiSpecExplorer::explore() instead. The workflow is inspired by Schemathesis, but the supported strategy matrix below is the contract; this package does not claim feature parity.
use Tests\TestCase;
use Studio\Gesso\Fuzz\ExploredCase;
use Studio\Gesso\Laravel\ExploresOpenApiEndpoint;
use Studio\Gesso\Laravel\ValidatesOpenApiSchema;
use Studio\Gesso\Attribute\OpenApiSpec;
#[OpenApiSpec('front')]
class CreatePetTest extends TestCase
{
use ExploresOpenApiEndpoint;
use ValidatesOpenApiSchema;
public function test_create_pet_contract(): void
{
$this->exploreEndpoint('POST', '/v1/pets', cases: 50, seed: 1)
->each(function (ExploredCase $case): void {
// Laravel parses form fields only from `parameters`, never from raw content.
$form = $case->contentType === 'application/x-www-form-urlencoded';
$this->call(
'POST',
$case->uri('/api'),
parameters: $form ? $case->bodyAsFormFields() : [],
cookies: $case->cookies,
server: $this->transformHeadersToServerVars([
'Accept' => 'application/json', 'Content-Type' => $case->contentType, ...$case->headers,
]),
content: $form ? null : $case->bodyAsString(),
)->assertSuccessful();
});
}
}What you get per case (Studio\Gesso\Fuzz\ExploredCase):
| Property | Description |
|---|---|
body | Generated body value (or null when the operation has no requestBody, or none the explorer can synthesize) |
contentType | The media type body was generated for: application/json or a +json type, or application/x-www-form-urlencoded when that is the only schema-carrying media type. multipart/form-data is never synthesized; a required multipart-only body fails loudly |
query | name → value for every in: query parameter |
headers | name → value for every in: header parameter (excludes the OpenAPI-reserved Accept/Content-Type/Authorization) |
pathParams | name → value for every {placeholder} segment |
cookies | name → value; empty unless a check populated it (cookie generation is not implemented). A dispatcher must forward these to its client's cookie bag — a Cookie: header alone never reaches $request->cookie(...) in Laravel or Symfony test clients |
method, matchedPath | The resolved spec template (/v1/pets/{petId}) and its method |
kind, targetKeyword, targetPointer | Valid/invalid classification and the single constraint targeted by a negative case |
expectedStatusClasses | Explicit response classes supplied for a negative case (for example [4]) |
seed, caseIndex | Stable replay identity |
The collection is Countable and IteratorAggregate, so foreach ($cases as $case) works too if you prefer it over the fluent each() helper.
Use $case->bodyAsString() to serialize the body in $case->contentType for a client that sends raw bytes; for JSON it is $case->bodyAsJson(), which encodes the original value without loss. Laravel's call() and Symfony's KernelBrowser::request() do not parse raw form content into the request's input, so pass $case->bodyAsFormFields() as their parameters for a form case instead, as above. bodyAsArray() is a lossy compatibility helper for array-only consumers: it recursively turns {} into [], including nested objects. Feeding it to Laravel's postJson() or Symfony's jsonRequest() can turn a generated valid object case into an invalid array. Do not use JSON_FORCE_OBJECT, which would also change genuine JSON arrays. bodyAsJson() encodes null as literal JSON null; when an operation declares no body or a check omits a required body, omit the raw content instead. Encoding failures throw JsonException.
Form-encoded bodies (application/x-www-form-urlencoded) go through bodyAsFormFields(): values are sent as strings (true / false, numbers verbatim), object-valued properties are JSON-encoded (the Encoding Object default for an object), null properties are dropped, and lists use PHP bracket notation (tags[0]=a&tags[1]=b), which is what parse_str() and HttpFoundation read back. encoding.<part>.style / explode are not applied by the generator. Every form case is checked the way the request validator will read it: a valid case that no longer satisfies the schema once form-encoded (a required list that must stay empty, a part encoding the generator does not apply) makes the operation fail loudly, and a negative-case mutation that becomes valid on the wire (a type mismatch such as "42" for a string, an empty list that drops out) is not used.
uri($prefix) substitutes and URL-encodes path parameters, prepends an optional application prefix, and appends the generated query string.
Explore response payloads against a generated SDK
OpenApiResponseExplorer exercises the other side of the contract: whether a generated SDK can decode every supported branch of one response schema and encode the values without losing data. Unlike request exploration's fixed case count and rotation, the response explorer derives enough cases to cover every branch of every reachable choice point, then appends any requested extraCases.
use Studio\Gesso\Fuzz\GeneratedResponseCase;
use Studio\Gesso\Fuzz\OpenApiResponseExplorer;
OpenApiResponseExplorer::explore(
'front',
'POST',
'/oauth/introspect',
200,
seed: 1,
)->each(function (GeneratedResponseCase $case): void {
$model = ObjectSerializer::deserialize($case->bodyAsObject(), IntrospectResponse::class);
$case->assertRoundTrip(ObjectSerializer::sanitizeForSerialization($model));
});Laravel tests using ExploresOpenApiEndpoint can call $this->exploreResponseSchema(...) with the same arguments except the spec name. If seed is omitted, response exploration uses the replayable default seed 0. See the SDK round-trip guide for the complete case contract, fidelity rules, deterministic replay, and unsupported response outcomes.
To replace one-test-per-response maintenance with a loud spec-wide gate, register explicit SDK mappings on OpenApiResponseExplorer::exploreSpec():
OpenApiResponseExplorer::exploreSpec('front', seed: 1)
->includeTags(['public'])
->mapResponse(
'introspect',
200,
static fn (GeneratedResponseCase $case): mixed => sdk_decode($case->bodyAsObject()),
static fn (mixed $model): mixed => sdk_encode($model),
)
->failOnUnmapped()
->assertRoundTrips();The plan uses the whole-spec tag/method/path/operation/deprecated filters and stable per-operation crc32 seeds. Exact, range, and default status mappings are supported; one mapping exercises every JSON media type for that response. The summary distinguishes explicit skips from decoder and round-trip failures, while failOnUnmapped() makes newly added unmapped schemas fail CI. Laravel tests call $this->exploreResponseSpec() for the same builder. See the SDK round-trip guide for callback and reporting details.
Explore a whole spec
exploreSpec() enumerates the Path Item operations defined by OpenAPI, including 3.2 additionalOperations, then generates and dispatches cases for every selected method supported by the explorer. It uses the same Laravel spec resolution as response validation (#[OpenApiSpec], openApiSpec(), then default_spec). The two traits are designed to be composed:
use Studio\Gesso\Fuzz\ExploredCase;
use Studio\Gesso\Fuzz\ExploredOperation;
use Studio\Gesso\HttpMethod;
use Studio\Gesso\Laravel\ExploresOpenApiEndpoint;
use Studio\Gesso\Laravel\ValidatesOpenApiSchema;
class ApiContractTest extends TestCase
{
use ExploresOpenApiEndpoint;
use ValidatesOpenApiSchema;
public function test_public_contract(): void
{
$summary = $this->exploreSpec(casesPerOperation: 20, seed: 20260711)
->includeTags(['public'])
->excludeOperations(['admin.users.destroy'])
->authenticateUsing(fn (ExploredOperation $operation) => $this->actingAs($this->userFor($operation)))
->dispatchUsing(function (ExploredCase $case) {
$form = $case->contentType === 'application/x-www-form-urlencoded';
return match ($case->method) {
HttpMethod::GET => $this->get($case->uri('/api'), $case->headers),
HttpMethod::POST => $this->call(
'POST',
$case->uri('/api'),
parameters: $form ? $case->bodyAsFormFields() : [],
cookies: $case->cookies,
server: $this->transformHeadersToServerVars([
'Accept' => 'application/json', 'Content-Type' => $case->contentType, ...$case->headers,
]),
content: $form ? null : $case->bodyAsString(),
),
default => throw new LogicException('Add the method to the test dispatcher.'),
};
})
->assertResponses(); // ValidatesOpenApiSchema auto_assert validates each response
self::assertFalse($summary->hasSkips(), $summary->skips[0]->reason ?? '');
}
}Framework-agnostic exploration
ExploresOpenApiEndpoint is Laravel-only; there is no Symfony equivalent. Symfony, Pest, and plain PHPUnit users call OpenApiSpecExplorer::explore() directly. It returns the same filter/hook builder, so everything below applies unchanged:
use Studio\Gesso\Fuzz\ExploredCase;
use Studio\Gesso\Fuzz\OpenApiSpecExplorer;
$summary = OpenApiSpecExplorer::explore('petstore', casesPerOperation: 20, seed: 7)
->dispatchUsing(fn (ExploredCase $case) => $this->sendRequest($case))
->assertResponseUsing(fn (mixed $exchange) => $this->assertExchangeIsValid($exchange));dispatchUsing() returns whatever your client produces and assertResponseUsing() validates it. The runnable examples/psr7 suite demonstrates this with OpenApiPsr7Validator assertions.
Filters and hooks
includeTags()/excludeTags()match when any operation tag overlaps.includeMethods()/excludeMethods(),includePaths()/excludePaths(), andincludeOperations()/excludeOperations()use exact values. Fixed HTTP methods are canonicalized; OpenAPI 3.2 custom method spelling stays case-sensitive.- Deprecated operations are excluded by default. Call
includeDeprecated()to opt in. setUpUsing()andtearDownUsing()run once per operation;authenticateUsing()runs after setup and before its cases.mutateCasesUsing()runs per generated case and must return anExploredCase. ItswithBody(),withQuery(),withHeaders(),withCookies(), andwithPathParams()helpers support credentials, stateful IDs, and other request-specific changes without mutating shared state.
Operations that are declared but cannot be generated are returned in SpecExplorationSummary::$skips with their reason. This includes schema-less required inputs and methods outside GET, POST, PUT, PATCH, DELETE, and QUERY. Malformed paths, Path Item, additionalOperations, and Operation Object nodes are spec errors, not unsupported operations: a structural preflight fails before any request is dispatched. Filters intentionally remove operations and therefore do not create skip entries. A filter set matching nothing fails loudly instead of producing a test that asserted nothing.
Replay and parallel runs
The global seed is deterministic. Each operation receives a stable derived seed based on (spec, method, path, global seed), so adding or reordering a different operation does not change existing cases. A dispatch, mutation, or assertion failure prints the spec, operation ID, method/path, both seeds, case index, and a minimal OpenApiEndpointExplorer::explore(...) replay expression. Each case also exposes replayToken(), replaySnippet($specName), and curlSnippet($baseUrl). The token identifies generation inputs; the PHP and curl output include the concrete generated request. curlSnippet() redacts sensitive header, query, and form-field values (Authorization, Cookie, API-key/token/secret/password-style names) as <redacted> so the command is safe to paste into CI logs; pass redactSensitiveHeaders: false to keep the raw values.
Summaries are local immutable values; the explorer adds no process-global aggregation. Run one plan per parallel worker partition and use the existing coverage sidecar/merge workflow to aggregate validated response rows.
Follow response links
Opt in with followLinksUsing() to chain operations connected by explicit OpenAPI response links:
use Psr\Http\Message\ResponseInterface;
use Studio\Gesso\Fuzz\OpenApiSpecExplorer;
$summary = OpenApiSpecExplorer::explore('front', casesPerOperation: 5, seed: 1)
->dispatchUsing(fn ($case, $operation) => dispatch_request($case))
->followLinksUsing(fn (ResponseInterface $response) => $response, maxDepth: 2)
->assertResponses();
// $summary->followedLinks counts the follow-up requests.After every valid case, each link declared on the observed response status is followed: Gesso generates one request for the target operation (seeded from the root case and the link names, so chains replay deterministically), applies mutateCasesUsing(), then overrides it with the link's parameters and requestBody evaluated from the parent request and response ($response.body#/id, $request.path.petId, …). The follow-up is dispatched and asserted through the same hooks, inside the root operation's setUp/authenticate/tearDown scope, and its own links are followed up to maxDepth hops (a (method, path) already in the chain is dispatched but not expanded again). Explicit exclusions (excludeMethods(), excludePaths(), excludeTags(), excludeOperations(), and deprecated operations unless includeDeprecated()) also apply to link targets; inclusion filters only select the chain roots. Sibling links targeting DELETE run last, and a pair is only checked when no other state-changing follow-up ran between the parent response and the follow-up.
followLinksUsing() receives whatever dispatchUsing() returned and must return a PSR-7 response or ['status' => int, 'headers' => array, 'body' => string].
Each (parent, follow-up) pair is checked:
LinkChainCheck | Fails when |
|---|---|
use_after_free | a 2xx DELETE is followed through a link by a 2xx GET of the same request path |
ensure_resource_availability | a 2xx POST/PUT/PATCH is followed through a link by a 404/410 GET |
A 202 Accepted parent is never checked: it only queues the work, so an immediate read-back proves nothing.
A violation throws immediately with the ordered chain (1. POST /pets → 201 (case 0, operation seed …), 2. [GetPet] GET /pets/1 → 404), the replay call for the root case, and a curl command for the last request. Chains are reported, not reduced: FailureReducer keeps its single-case contract.
Links are opt-in: plans without followLinksUsing(), specs without links, and negativeCases() plans behave exactly as before. A link with both or neither of operationId/operationRef, an external operationRef, an unknown target, a target method the explorer cannot dispatch, an ambiguous or unknown bare parameter name (qualify it as path.id, query.id, …) or an expression that does not resolve fails loudly instead of being skipped.
Safety for mutating operations
Whole-spec exploration can execute POST, PUT, PATCH, and DELETE. Run it only against an isolated test database or disposable environment, wrap each case/operation in your framework's transaction reset, and start with includeMethods(['GET', 'QUERY']) when evaluating an unfamiliar spec. Use includeOperations() for a deliberate allowlist when an endpoint triggers external effects that a database rollback cannot undo.
Generation behaviour
Every purportedly valid value is checked against the converted JSON Schema before dispatch. A generator bug therefore fails locally with an Internal fuzz generator defect diagnostic instead of sending invalid data to the API.
| Strategy | Valid generation | Targeted invalid mutation |
|---|---|---|
| Scalars | type, const, enum, nullable branches | wrong type, const/enum miss |
| Strings | min/max length, common regex patterns (including anchored character classes with fixed or + quantifiers, FQDN labels with a fixed domain suffix, and phone-number alternations), Unicode code points, Faker-backed email/UUID/date/time/URI/host/IP formats | below/above length, pattern miss, invalid recognized format |
| Numbers | inclusive/exclusive bounds and multipleOf, including OAS 3.0 boolean-exclusive lowering | outside/equal-exclusive bound, non-multiple |
| Arrays | items, prefixItems, min/max items, uniqueItems | too few/many or duplicate items |
| Objects | properties, required, min/max properties, schema-valued/default additional properties | missing required, extra forbidden, too few/many properties, nested property constraint |
| Composition | Request cases rotate oneOf/anyOf and conditional branches. Response payload exploration guarantees at least one case for every branch of every reachable composition, nullable, and optional-property choice point. Both merge allOf/range assertions and preserve lowered discriminator branches. | deterministic composition miss where one can be isolated |
Arbitrary regex synthesis, recursive schemas, contains, patternProperties, dependentSchemas, and unevaluated* generation are not currently strategies. Those keywords remain validator features; an operation whose valid value cannot be synthesized fails locally with a supported-subset diagnostic or is an explicit whole-spec skip carrying that reason.
Branch-complete response enumeration is bounded to 32 nested property/item levels, 256 collected choice points, and 10,000 visited schema nodes across all branch contexts. Exceeding any limit fails locally with a supported-subset diagnostic; Gesso never returns partial branch coverage.
- Request cases alternate optional object properties between included and omitted. Response payload exploration pins both presence branches for every reachable optional property.
- Required keys are always emitted.
- Path resolution accepts both the spec template form (
/v1/pets/{petId}) and concrete URIs that match it (/api/v1/pets/123withstrip_prefixes=/api). Captured URI values are intentionally discarded —pathParamsis always regenerated from the operation spec for consistency.
Memory for large schemas
The explorers check every value they generate against its schema, and keep the schemas they have checked against parsed so the next check does not parse them again. fuzz.cache_memory_mb in gesso.php is how much memory those parsed schemas may hold, 16 MB by default:
'fuzz' => [
'cache_memory_mb' => 64,
],Raise it when exploreInvalid() is slow on an operation with a large request body. One call checks its cases against about a hundred variants of the body schema, in the same order for every case, so it is fast when all of them fit and gains little when only some do. In GitHub's REST API description 16 MB holds them for 319 of 321 request bodies; the two largest, 38 KB of schema each, need about 60 MB (measurements).
- The budget is per process. Under a parallel runner every worker has its own.
- In a Laravel test the application's value is the one in force: the service provider applies
config('gesso.fuzz.cache_memory_mb')on every boot, after the PHPUnit extension. A package tested with Testbench has no applicationgesso.php, so set that config value in the test's environment setup. - It counts toward PHP's
memory_limit, on top of what the run already uses. - Schemas over the budget are dropped, least recently used first, and parsed again when they come back. The generated cases are the same at any value;
0keeps only the schema checked last.
seed and determinism
When fakerphp/faker is installed, generation uses Faker's locale-aware primitives and is deterministic for a given seed:. Date and time formats are drawn up to 2020-01-01T00:00:00Z and written in UTC, so the clock and the default time zone do not change them. Without it, ordinary strings and scalar boundaries use deterministic counter-based values. Supported character-class patterns sample multiple class members after the first boundary case, so increasing cases: also increases their input variety. Recognized formats such as email and UUID cannot be synthesized reliably by that fallback: the existing one-shot warning is followed by the valid-case self-check, so the operation fails locally rather than dispatching a value that does not satisfy its format.
# Required when explored schemas use Faker-backed formats
composer require --dev fakerphp/fakerNegative cases and reduction
Negative exploration requires the expected response class. There is no implicit "anything except 5xx" fallback:
$this->exploreInvalidEndpoint(
'POST',
'/v1/pets',
expectedStatusClasses: [4],
cases: 20,
seed: 7,
)->each(function (ExploredCase $case): void {
$form = $case->contentType === 'application/x-www-form-urlencoded';
$response = $this->call(
'POST',
$case->uri('/api'),
parameters: $form ? $case->bodyAsFormFields() : [],
cookies: $case->cookies,
server: $this->transformHeadersToServerVars([
'Accept' => 'application/json', 'Content-Type' => $case->contentType, ...$case->headers,
]),
content: $form ? null : $case->bodyAsString(),
);
self::assertContains(intdiv($response->getStatusCode(), 100), $case->expectedStatusClasses);
});For a whole spec, add ->negativeCases([4]) before dispatchUsing() and inspect the same metadata in assertResponseUsing(). Each generated invalid case is self-checked to ensure it actually fails the complete schema.
FailureReducer::reduce($case, $classify) deterministically removes body members only while the callback returns the original non-empty classification. Use a stable value such as status:500 or an exception class. Reduction is deliberately classification-preserving; it never equates every failure.
Named contract checks
Schema validation cannot see protocol-level contract holes — an ignored security requirement, a required: true header nobody checks, an undocumented method that answers 200, invalid input that is accepted, valid input that is rejected. The opt-in ignored_auth, missing_required_header, unsupported_method, negative_data_rejection, and positive_data_acceptance checks probe those directly and live in Studio\Gesso\Contract\. See named contract checks.
Remaining gaps
- Arbitrary ECMA-262 regex synthesis and recursive/reference-cycle generation.
- Cookie and
parameters[].contentfuzz generation. - Structural shrinking inside nested arrays/objects; current reduction removes top-level body members.
- Measurement-based feature parity with Schemathesis.
- Stateful chains beyond explicit
links: parameter-name inference andLocation-header learning (#400 phase 2).