aspace://architecture/start

How to read this guide

This guide is for the person who decides how a system on AgentSpaces is shaped: how many peers, where they run, which boundaries the application draws, what each space promises, and where the fleet meets the rest of the enterprise. It assumes some working knowledge of distributed systems. It's a companion of the developer guide for the API and the workflow guide for the stage-by-stage patterns; this guide decides what those two build.

We designed AgentSpaces to bookend a story that began at Bell Labs in 1986 with the publication of Linda, continued on through the partnership with Sun Microsystems and their release of Jini and JXTA. When we arrived at Lucent in 96 and '00, we reviewed, designed, and prototyped with Jini and JXTA, though didn't wind up including it on any of the product lines we architected. We're ultimately coming full circle as the Bell Labs crew comes back together again at Bad Monkey to combine the best parts of Jini and JXTA with a healthy dose of modernization on related standards (PKI, TLS, mTLS, DTLS, CBOR, etc). While P2P found a home in CDNs and other large-scale economic saving endeavors, technical improvements have been driven by a steady state of research in the intervening years. If we implement one of those papers, we'll cite it in the documentation.

What AgentSpaces is, in one sentenceA fabric of signed, leased entries replicated across the peers of a group, with coordination services on top of that replication, so that agents on any host, in any language, coordinate through shared time-boxed memory rather than through a broker or a database. Everything in this guide follows from three properties: every entry is signed and leased, every peer holds a full replica of the spaces it registers, and every verb is a protocol the peers run among themselves.
aspace://architecture/picture · chapter 0

The system in one picture

ONE GROUP: EVERY PEER HOLDS A REPLICA OF EVERY SPACE IT REGISTERSpeer-aSpring Bootintakedecisionsledgeragents, cardscapabilities on pipespeer-bMicronautintakedecisionsledgeragents, cardscapabilities on pipespeer-cPythonintakedecisionsledgeragents, cardscapabilities on pipespeer-dconsole, read-onlyintakedecisionsledgeragents, cardscapabilities on pipesgossipgossipgossipsigned entries, leases, anti-entropy, SWIM membership, one tick clockWHAT A PEER IS MADE OFidentitypeer key, certificatespeeringtransports, SWIM, gossipdiscoverycards, ads, semanticspacesreplicated tuple spacescapabilitiesvote, aggregate, log, learnthe programming model: annotations on plain objects, bound by AgentBinder; the same verbs from Python, TypeScript, and Clojure
The system in one picture. A group is a set of peers that gossip; every peer that registers a space holds a full replica of it, so there is no broker, no primary, and no database between agents. A peer is five layers under a programming model; a Python or TypeScript peer is the same wire with the same layers up to the ordered log.

A peer is one process running the library. It has an Ed25519 identity, joins one or more groups, and registers the spaces it takes part in. A group is the set of peers that gossip with one another: membership by SWIM probes, advertisements by a gossip stream, space state by anti-entropy. A space is a replicated tuple space: typed entries with leases, written, read, taken under a lease, and completed, all as signed records folded into a CRDT so that every replica converges whatever order the records arrive in. Capabilities (votes, push-sum aggregates, the ordered log, semantic discovery, gossip learning) run on pipes between peers of the group. Agents are ordinary objects bound to spaces and capabilities by annotations, or by the equivalent decorators in Python and TypeScript and the defagent form in Clojure; each publishes a signed card the others discover.

Four consequences shape every architecture on it. There is no primary and no broker, so there is nothing to fail over and nothing to scale separately; a peer that registers a space holds all of it. Convergence has a rate, set by one tick clock per peer (250 ms by default) and a gossip period (1 s) carried in the group's advertisement: a write reaches every replica in a few periods, and the settle windows, leases, and deadlines of chapters 6 to 9 are multiples of that rate. A guarantee stronger than replication (one taker per entry, one completion fleet-wide, a decision) is a protocol run on demand on the spaces that need it, so each space stands on its own rung of the ladder in chapter 7. And identity is on the wire: each record carries its issuer's signature, so the trust perimeter is membership and admission rather than a network edge.

Where the pieces are describedThe layers and their APIs: the developer guide's chapters on the Space API, discovery, capability services, peering and transports, and security. The formats and algorithms: SPEC.md and TECH-SPEC.md (section 2 of the technical specification, "Why these protocols", is the rationale this guide builds on).
aspace://architecture/sizing · chapter 1

How many peers, and what runs where

Count peers by what they hold; the number of agents is a separate question. An agent is a binding on a peer, and a peer can host as many agents as its process can run; three peers can seat thirty agents. A peer is a unit of replication (a full copy of each space it registers), a unit of failure (its take leases lapse together), a unit of trust (one identity, one keystore, one channel), and, for the ordered log, a unit of quorum. Decide the peer count on those four axes and let agent placement follow.

Three sizes

SizeWhat it looks likeWhat changesWhere it is shown
3 to 5 peersone site, every peer listens, every member is a seednothing special: full views, full replicas, one ordered log needs a majority of 2 of 3 or 3 of 5examples 02, 12, 18 (three peers, five spaces); the Spring, Micronaut, and LangChain4j example fleets
5 to 15 peersa site or two, a rendezvous, roles emerge (console, gateway, relay)advertisement caches fill (8,192 entries by default, 32,768 on a rendezvous); keep card TTLs and refresh at the defaults; decide which few peers hold the ordered log's member setthe Party Bus: 15 peers, 9 spaces, 34 agents on TLS
Beyond thatseveral sites, many dial-only workers, partner groupsreplication is full by default, so put a large space on a tag-sharded replica only on the peers that need it (builder-only today); split by group rather than by space when the perimeter differs; expect no published scale ceiling, measure with the stress harnesses in agentspaces-perf and the traffic auditsa fifty-member simulated fleet in the peering suite (RobustnessScaleTest) idles at about five frames per member per tick and drops a crashed member from every view within its TTL; six real peers idle at 0.7 frames per peer per second, the same across two hosts (examples/superfriends/two-host-audit.sh, PERF2 §5)

The library publishes no peer-count limit. Three things grow with the fleet: membership views (full; every peer knows every member), replication (full per registered space; every replica holds every entry until its tombstone collects), and the advertisement cache (bounded, evicting on read). Gossip propagation is O(log N) rounds, and a per-issuer token bucket (50,000 frames, 10,000 a second) bounds what any one peer can push. The one cost that is linear per member is the advertisement refresh, which goes directly to every member: once a fleet outgrows what the probe cycle can keep alive inside the member TTL, the refresh runs at half the TTL and is the keepalive, so an idle member costs about (N - 1) frames per refresh interval; probes and anti-entropy back off while nothing moves. None of these is a cliff at ten or fifty peers, but each is a reason to keep a group to one purpose and to use several groups for several purposes.

Roles

Two roles exist on the wire, and four more are conventions you adopt. RENDEZVOUS is a peer that keeps a large advertisement cache and answers discovery queries directly; a site seeds from it. RELAY is a peer that forwards a fully signed frame from one member to another, one hop only, so that two dial-only peers can reach each other; relays never re-relay. By convention a console peer is a read-only member that serves the operator's view and, with the directive grant, command-and-control; a gateway peer fronts the fleet for A2A or MCP clients; a coordinator peer holds an ordered log's leadership most of the time, though any member of the log's set may lead; a beacon peer is a rendezvous that also serves the signed beacon document of the group's live seeds, so that newcomers have a place to read them.

DecisionRuleWhy
Workers per kindthree or four, on different peerseach added independent worker cuts the one-of-N failure probability again, so past that point the floor (shared provider, shared seed) usually sets availability; there is no universal cutoff, see chapter 6
Ordered logsone per group per peer, a fixed member set of three or fivea quorum is a majority; reconfiguration means a new log; keep the set on the peers with the steadiest uptime
Rendezvousone per site, two where the site must not lose discoveryit is a cache, never an authority; losing it slows discovery and changes no result
Beaconone per site, two where the site must not lose discovery; its own keystore; TLS in front of its HTTP portit serves a signed document of the group's live seeds and holds no secret; losing it slows newcomers and changes no result
Console and gatewaytheir own peers, read-only or edge-facingthey hold no take leases, so their restarts cost the flow nothing, and their inbound ports are the only ones you expose
Dial-only peersas many as you likea peer with no bind advertises no endpoint, speaks only to what it dialed, and needs a relay to speak to another dial-only peer
Keep in mindEverything runs on one clock, somewhat inspired by game engines and the notion of a tick of the clock. The clocks are injectable, and we change them in the test harness to speed things up in certain cases. The tick drives the SWIM probe, the self-advertisement, anti-entropy, and every capability's protocol round; the gossip period is carried in the group advertisement and is the same for every member. Changing either changes the convergence rate of the whole fabric, and with it the right settle windows and deadlines. Typically the default should be left as is, but you can set them once, early, and derive the rest.
aspace://architecture/discovery · chapter 2

Discovery and bootstrap

A. ONE SITE, FULL MESHpeerpeerpeerpeerpeer3 to 15 peers; seeds are any memberB. SEEDS AND DIAL-ONLY PEERSseedlistens, tcp or tlslaptopno bindpythonno bindfield boxNATrelayrole RELAYdial-only peers reach each other through a relayC. TWO SITES, RENDEZVOUSrendezvousrole RENDEZVOUSsite 1peerpeersite 2peerpeereach site seeds from the rendezvous;a cache, never an authority
Three topologies that cover most deployments. A: one site, every peer listens and any member is a seed; views and replicas are full. B: a few listening seeds and many dial-only peers (laptops, Python and TypeScript clients, boxes behind NAT); a RELAY role lets two dial-only peers reach each other in one hop. C: sites behind their own networks seed from a RENDEZVOUS peer, which is a large advertisement cache and nothing more; losing it slows discovery and breaks nothing.

The options

A group's id comes from one of two places. With a founding text, you give every peer the same string and each derives the same group id locally; no signed document exists and nobody holds a secret. This is what the starter's founding: does and what every example uses, and it fits one organization, where the operator controls every peer's configuration. With a signed founding, a founder signs a founding document that the id is a hash of, and members hold it (founding-document, or a group founded in code). A peer that has only the id, as join: aspace://<id>, dials a seed, asks for the document, verifies that it hashes to exactly the id it wanted, and only then introduces itself; a hostile seed can delay that join and can never substitute a policy. A group started from a text has no founding to serve, so a peer joins it by receiving the text, and a peer that tries to join it by id waits for the join timeout and fails. The signed founding fits an id that travels further than the configuration: a partner's setup, a printed credential, a registry.

MechanismWhat it givesWhat it costsWhen
Seeds (groups[].seeds)a way in; a member once joined needs them no morea list to maintain; a seed that is down is only a delay if there is anotheralways; give every peer at least two
Rendezvous rolea large advertisement cache that answers queries directly and seeds a whole siteone more peer to run; none to correctnessa site, a WAN, any fleet past a handful of peers
Relay roleone signed hop between two peers that cannot dial each othera relay peer per region; relayed frames are refused under required attestationdial-only peers that must reach each other; NAT on both ends
Multicast beacon (agentspaces.multicast.*, off by default)LAN peers find a seed without configurationUDP on the segment; only a hint under required attestationa lab, a vehicle, a floor of devices on one segment
Join by group ida verified founding document before the first framea reachable member that holds the signed founding (a group started from a text has none); a timeout (10 s by default)ids that cross an organizational line
A DNS name with several addresses (seeds naming a host)every address of the name is dialed, not the firstnothing; on by default since 0.3.0three seeds behind one name, Docker Compose, a headless service
SRV, DNS-SD, a TXT pointer, a well-known HTTPS document (agentspaces.bootstrap.*, the agentspaces-bootstrap project)the enterprise's own directory answers "whom do I dial first": Kubernetes, Consul, Cloud Map publish SRV; the search domain DHCP hands out carries _agentspaces._tcp; a beacon document lists the live seeds and carries the founding when the group has a signed onerecords to register (the beacon prints them); a locator re-asked only when the view is empty and its hint has agedany fleet past one site; pods whose seeds move; partners who hold only an id and a URL
A beacon (agentspaces-beacon)one URL where newcomers read the group's current seeds, signed by the beacon; DNS needs only a pointer recorda stable peer with its own keystore, TLS in front of its HTTP port, and records registered by handa seed set that changes, partners, any fleet whose seed list went stale

There is no universal bootstrap group and no mDNS; a deployment that needs a directory of groups founds one and seeds from it, or runs a beacon (agentspaces-beacon) and registers its records in the DNS it already administers. Every mechanism above is a locator: it answers where to dial, and the fabric decides who is admitted, so a poisoned DNS answer or a hostile network's search domain can delay a join and never substitute a group when the id is configured. A group identity is accepted from a locator only when the operator names it, by default only a beacon document fetched over HTTPS or read from a file. The rendezvous and relay roles are declared by the peer that plays them (agentspaces.roles) and learned by everyone else from its advertisement.

Three of these together are a shape readers may know from elsewhere. A relay that forwards what it cannot read, under a byte budget (RelayPolicy.dailyBudget), is the DERP of Tailscale or the TURN of WebRTC with the cost line as a property; a beacon that holds no secret and admits nobody is a coordination server; a wss listener on 443 behind an ingress completes the analogy for browsers. The difference from those systems is where admission lives: the fabric decides who is a member, and every locator and relay only answers where to dial or carries what it cannot read. Chapter 11 ranks the architectures this opens.

The chain: one ordered list of locators for each group

newcomerempty viewasksTHE GROUP'S CHAIN, IN ORDERseedsconfigured list, names expandeddns-sdPTR, SRV, TXT under the domaindns-txt, url, documenta pointer to a beacon documentdns-srv, k8sSRV records at a name you choosemulticastwhat LAN datagrams announcedmerge in chain orderdrop duplicatesexpand names lastkeep the first foundingshortest lifetime winsone hintdial, check, jointhen gossip takes overasked again while the view is empty,30 seconds to 10 minutes apart
A chain turns several directories into one list of addresses. A newcomer with an empty view asks its group's chain. Every locator that applies answers at once under its own deadline; a locator with nothing, or one that times out, contributes nothing. The chain merges the answers in order, drops duplicates, expands names to addresses, and returns one hint. The newcomer dials the addresses as it would dial any stranger, checks a founding when it has one, and joins.

Every group has a chain, the ordered list of locators its peers ask when they join and whenever their view of the group is empty. A locator answers with a hint: addresses to dial in priority order, and optionally the group id it believes, the signed founding, a lifetime, and the evidence that produced the answer. The chain asks every locator that applies to the group at once, each under its own deadline (3 seconds by default). A locator that fails or times out contributes nothing and stops nothing. The chain merges the answers in chain order, removes duplicates, keeps the first founding it finds, expands names to addresses last, and gives the merged hint the shortest lifetime among the contributors. With first-wins it asks one locator at a time and stops at the first that returns addresses. The default order starts with seeds, so the configured list ranks first, and with first-wins a non-empty list means that nothing later is asked.

A peer asks again while its view is empty, no sooner than refresh-min (30 seconds) and no later than refresh-max (10 minutes). require fails startup, naming the group, when the first ask answers nothing, and trace logs every locator's evidence. Chains belong to groups because groups are found in different ways: a site group through DNS, a partner's group through a URL. Each group's bootstrap block overrides the fleet-wide one, and the Spring Boot starter and Micronaut compose each chain from those keys and from any BeaconLocator bean the application defines. The developer guide's bootstrap section has the API.

A chain suggests addresses, and the peer decides what to believe. Every address is dialed as a stranger's would be, with the handshake, the signature checks, and the group's admission policy. When a peer joins by id, it checks the founding against the id. A locator's claim about the group id counts only when the locator is in accept-group-id-from, which defaults to url and document. Naming a plain-DNS locator there stops startup unless insecure-group-id is set or the host's resolver authenticates its answers, as the DNSSEC resolver in agentspaces-bootstrap-dns does. beacon-peer pins the PeerId that must have signed any document.

Beacons: running something to be found

The word beacon names three things, and an architect should keep them apart. A beacon is a peer that newcomers can find and dial first: an ordinary member with a stable address, usually playing RENDEZVOUS, with no special role on the wire. The beacon document is the signed JSON that such a peer serves at /.well-known/agentspaces/beacon.json. The multicast beacon is the opt-in LAN datagram, a core feature that appears in a chain as the multicast locator.

The agentspaces-beacon project is a ready-made beacon. It joins a group, builds the document from its live view of the members (itself first, then the rendezvous members, then the other members that listen, up to eight), signs the document with its identity, and serves it with a cache lifetime, an ETag, read-only CORS, and a per-source rate limit. It prints the DNS records that point to the document. It holds no secret: the document never carries a content key, a credential, or a membership hint, and the group's own admission policy decides who joins. Six decisions shape how you run one:

  • Give it a keystore. The beacon's PeerId signs the document, and readers pin that PeerId with beacon-peer. Without a keystore the id changes at every start.
  • Put TLS in front of it. The document is public data, and a reader trusts the hostname.
  • The document follows the fleet, and DNS does not. SRV records registered by hand name the seeds of the day someone typed them. When the seed set changes, register only the pointer record (agentspaces.<domain> TXT url=) and let the document carry the seeds.
  • Run two where discovery must survive a loss. The second beacon joins the same group, and each document lists the other. A member that has joined needs no beacon, so a lost beacon slows newcomers and changes no result.
  • Python and TypeScript read the document by URL. Their bootstrap(url) joins a group whose entry carries a signed founding. A group started from a text has none, so those clients take the seeds from the document and join with the group id derived from the text.
  • A site with no route to the beacon reads a copy. The beacon writes the document to a file with --write-to, and the document locator reads a file or a classpath: resource again at every ask.

The beacon README walks one fleet through it, and the developer guide's beacon section has the classes for building your own.

Which mechanism for which network

A. CAMPUS: DNS AND A BEACONcorp DNSPTR, SRV, TXTbeaconserves documentlaptopdns-sdseedsmembers1 look up23 the documentlists the seeds4register the records once;the document follows the fleetB. KUBERNETES: HEADLESS SERVICEService agentspaces-seedsheadless: one SRV record per podseed-0seed-1seed-2worker podk8s locatorasks the cluster's DNSno beacon is needed: thecluster's DNS is the directoryC. TWO ORGANIZATIONSchecks the foundingagainst the id,pins the signerour peerjoin: aspace://idpartner beaconserves the documentdocumentfounding and seedspartner seedsmembers12 lists3 dialtrust rests on the idand the pinned signer
Three networks, three directories. A: a laptop on the campus finds the DNS-SD records under its search domain, follows the TXT url= to the beacon's document, and dials the seeds it lists; the records are registered once and the document follows the fleet. B: on Kubernetes the cluster's DNS already publishes one SRV record per pod of a headless service, so a worker pod asks it directly and no beacon is needed. C: across two organizations a peer holds only the group id and the document's URL, checks the founding against the id, and pins the signer.

Pick the locator by the directory the network already has. The table runs from the smallest network to the least trusted one.

NetworkWhat answers "whom do I dial first"ConfigurationFailure model
One LAN segment: a lab, a vehicle, a floor of devicesthe multicast beaconmulticast.enabled, and multicast.seal to seal the datagram under the group's content keyUDP on the segment only; under required attestation the datagram is a hint and the dial still attests
Docker Compose, or three seeds behind one namethe DNS name with several addressesseeds: ["seeds:7500"]; name expansion is on by defaulta dead address costs a delay while the others answer
Kubernetesthe cluster's DNS: the SRV records of a headless servicebootstrap.k8s with the service, namespace, and port name, or bootstrap.dns-srv with the owner namea rescheduled seed is a changed SRV record that the next ask finds; no beacon is needed
An enterprise campus, its laptops, and its sitesDNS-SD under the search domain DHCP hands out, a TXT pointer, and a beacon's documentbootstrap.order: [dns-sd] or [dns-txt] with domain: auto; a beacon per site; the printed records registeredoff the network the lookup finds nothing, and the peer logs one line and dials nothing; a poisoned answer delays a join and substitutes nothing when the id is configured
Two organizationsthe partner's beacon document and the group idjoin: aspace://<id>, one seed (seeds: [ "beacon.partner.example:7500" ]), bootstrap.order: [ seeds, url ], bootstrap.url, and bootstrap.beacon-peer; the group needs a signed founding, and the starter needs the seed (or a BeaconLocator bean) because for a join: group it counts only seeds, BeaconLocator beans, and multicast as locators, and the locators built from bootstrap.* do not countthe pin refuses a replaced document, and a hostile seed cannot show a founding that hashes to the id
A site with no route to the outsidea copy of the documentbootstrap.document: a file path or a classpath: resourcethe seed list is as fresh as the last copy
Browsers and edge clients behind HTTPSthe hub's document over HTTPS, then wssthe TypeScript client's bootstrap(url), or beacon.fetchBeaconDocument for a group started from a textread-only CORS lets a browser read the document, and a document whose seeds are all tls names nothing a browser can dial
A distrusted networkany of the above, with trust placed in the id, the pinned PeerId, and the attestation of every connectionrequire-attestation, beacon-peer, and a DnsResolver that validates DNSSECan unattested link is refused; a locator that is not in accept-group-id-from cannot name the group

Chains combine these. A laptop that roams between the campus and a partner's site carries two groups, each with its own chain: research-fleet through DNS-SD on the campus, and partner-fleet through the partner's URL.

Cards and advertisements

Every discoverable thing is a signed, typed, time-limited document: a peer advertisement (TTL 10 minutes), an agent card, a space advertisement, a capability advertisement (15 minutes each), the group advertisement (its founder sets the lifetime: the specification recommends 24 hours, and a group the starter founds from a text carries 365 days). They travel on one gossip stream, every cache keeps one per id and replaces it only with a newer issue, and a refresh cadence (card-refresh-millis, 5 minutes by default) keeps live things live. Revocations travel the same way and are never cached: what a replica already holds stays, and what arrives after is refused. Semantic discovery ranks cards by an embedding (a deterministic hashing embedder by default, a real model when you give one) and asks every rendezvous member over a pipe for theirs, or three sampled members when the group has no rendezvous; it is a routing hint, never a roster.

The design rule that follows: route work by entry type and treat a discovered name as a hint. An agent card tells a planner what is available and a remote caller how to address it; the space and the record type are what actually deliver the work, and they do so whether or not the card has arrived yet. A fleet whose correctness depends on a discovery result has put an eventually consistent cache on its critical path.

The starter's founding, and what it reservesA group the starter founds from a text is OPEN by default, uses the default gossip parameters and SWIM configuration, and is founded by a generated key the starter does not keep. Operations the specification reserves to the founder, revoking a peer and rotating the content key without a grant, therefore need either a group founded in code with a kept founder key (PeerNode.joinGroup(SignedGroupAdvertisement, ...)) or the grants the starter does expose (security.grants.key-rotator, the founder-or-ca revocation validator). Decide which before the fleet has data in it. The starter also takes membership: invite, under which the founder whose credentials admit is the peer named in founder (this peer when that is unset) and every other member presents a founder-issued or delegated credential in join-credential; you wire the POLICY membership in code today. See the developer guide's security chapter.
aspace://architecture/mixing · chapter 3

Mixing peers: hosts, languages, profiles

One group may hold peers on every host the project supports: the Spring Boot starter, Micronaut, LangChain4j on a plain JVM, Embabel, a peer built from the core API, and the Python, TypeScript, and Clojure clients. They share one wire (CBOR, Ed25519, wire version 3, pinned by golden vectors every client runs), so the question is never whether they can talk but what each can host. Decide placement by the table, then write each agent in the language its team writes.

Peer kindHostsDoes not hostPosture
JVM host (Spring, Micronaut, LangChain4j, Embabel, core)every space strategy, every capability including the ordered log, A2A and MCP gateways, the console, model servinglistens or dials; TLS by default (mtls)
Clojure (agentspaces-clj)the same as a JVM host through fleet/start and defagent, including ordered takes and joinsthe same
Python clienttakes, notifies, bids, ballots, decisions, proposals, estimates, LOCAL joins, push-sum, semantic queries, remote actions, ordered takes through OrderedLog as a Raft client, its own agent certificatean ordered-log voter; the decorators for ordered takes and reduces; the LEASED and ORDERED join modes; gossip learningdial-only on plain TCP; sends capability frames only to members it can reach
TypeScript clientthe same as Python, plus @orderedTake, @spaceReduce, and the LEASED and ORDERED join modes; on a Node service host (@agentspaces/node), an ordered-log voter and listenersgossip learning; an ordered-log voter, a listener, or the relay role anywhere but the Node service hostdials tcp on Node and ws, wss, http, and https wherever the runtime has a WebSocket or fetch; the Node service host listens on tcp and ws

Three rules keep a mixed fleet coherent. Profiles must meet. Note that a Python client speaks plain TCP, so the peer it dials must listen on TCP, while a TypeScript client (0.4.0) dials tcp, ws, wss, http, or https on Node and a browser member dials the same without tcp: either the fleet runs dev-local (signed frames, no channel encryption) or a JVM peer fronts the clients with a TCP listener while the JVM core stays on TLS; a mixed fleet degrades to its lowest rung unless require-attestation refuses the plaintext link outright. We expect 1.0.0 to provide the same transport options universally across all libraries, we started with Java so it proves the architecture and then we uplift python, typescript, etc. Once-only work needs the ordered log's voters. A Python or TypeScript worker takes under the lease race and completes atomically like any other, and it can take through the ordered log as a client of the voters (Python through OrderedLog, TypeScript with @orderedTake). The voters run on JVM hosts, or on a TypeScript Node service host, so a stage that needs the ordered log keeps its log on one of those, and the clients write into that space and take from it through the log. Capabilities announce themselves. A peer that runs an aggregate advertises it, and shares go only to members whose advertisement is live, so a bare client or a console never swallows mass; a client that wants to contribute starts its aggregate and advertises like everyone else.

The proofCouncilFleet in example 02 runs a Java coordinator with Python and TypeScript council workers in their own processes: tasks taken, a vote decided at a quorum of three, a push-sum epoch settled at the fleet mean, a semantic query answered over the pipe, a remote action served. run-trilingual-demo.sh plays it as its second act. The developer guide's language chapter has each client's surface.
aspace://architecture/boundaries · chapter 4

Spaces as lanes, groups as perimeters

GROUP: CLAIMS (ONE TENANT, ONE MISSION)intakeLEASE_RACEassessmentAUCTIONdecisionsvotespaymentsORDERED logledgerclosing records, long leasesone strategy and one retention per space:a lane is a space, as a queue would beteam A peersregistrars, screensteam B peersadjusters, panelops peerconsole, gatewayadmission per space: GROUP, ALLOWLIST, CREDENTIAL, AUTHORIZERGROUP: PARTNER (ANOTHER ORGANIZATION)commitmentsORDERED, content key sealedcatalogcards and assetspartner peerstheir hosts, their keysbridge peera member of both groupsgroup=claimsgroup=partnermembership policy OPEN, INVITE, or POLICY (DID)
Spaces draw lanes, groups draw perimeters. Inside one group, give each lane of the application its own space, since strategy, admission, and retention are per space. Across tenants, missions, or organizations, use groups: membership is the trust perimeter. A peer may belong to several groups and bind an agent into each by name, which is how one organization's flow hands commitments to another's without sharing a replica of everything.

One space per lane

A space carries one strategy (LEASE_RACE or AUCTION; the ordered log is a coordinator registered on a space rather than a third strategy), one admission rule, one replication mode, one settle window, and one retention policy through its leases. So the unit of design is the lane: give each stage of the application its own space the way you would give it its own queue, and let the record types say what flows. Example 18 runs a nine-stage claims flow over five spaces (intake, assessment, decisions, payments, ledger), each with the strategy its stage needs: the assessment space is an auction because adjusters price work, the payments space hosts the ordered log because a payment happens once, and also holds the reduce accumulator that keeps each claimant's running exposure beside the payments, and the ledger holds the closing record of every case under long leases.

Draw a new space whenKeep one space when
the strategy differs (one lane races, another auctions)the records are stages of one case and the same workers read them
admission differs (a lane only an attested agent may write)a reader wants one template over the whole lane
retention differs (hours for tasks, days for an audit trail)a join draws parts from them (a part may name another space; the join's own space holds its tickets)
the ordered log must live there (one log per peer per group)the volume is small enough that a full replica everywhere costs nothing

Within a space, tags and filters partition without a new space: a template matches on tags before it decodes a payload, so a keyless relay can route by tag, and a where filter narrows a worker to the entries addressed to it. Example 18 deploys one shop class per region on one space. Reach for a tag shard (TAG_SHARDED replication, builder-only today) when a space is large and most peers need one slice of it.

A group per tenant, mission, or conversation

A group is the membership perimeter: who gossips, who sees the advertisements, whose signatures count. One peer may belong to several, and an agent binds into a named group (group = "..." on every annotation, or every satisfying group when unnamed) with one card published per group. Draw a new group when the set of parties differs: a tenant, a mission with its own lifetime, a partner organization, a conversation that should leave no trace in the others. Draw a new space when only the lane differs. The bridge between two groups is a peer that is a member of both and hosts an agent in each; it hands records across by taking in one and writing in the other, and it is the one place a cross-group policy can be enforced.

BoundaryMechanismStarter support
Who may joinmembership policy OPEN, INVITE (a founder-signed credential), POLICY (a validator SPI, for instance DID)membership: open and invite (with founder and join-credential); POLICY you wire in code
Who may write or take a spacespace admission GROUP, ALLOWLIST, CREDENTIAL, AUTHORIZER, judged per acting agent; reads stay open to the groupspaces[].admission, allowed-agents, credential-issuer
Who may read a payloada group content key (AES-256-GCM); members without it relay and cannot read; rotation with a cutover delaygroups[].content-key and content-key-rotation
Who may do an operationthe authorizer's eleven operations, ten of them as lists under security.grants.*, or OIDC scopesten grant lists; OIDC scopes under the OIDC profiles
Keep in mindTypes, tags, issuers, and leases stay plaintext even under a content key: the fabric routes on them. If the metadata itself is sensitive, the boundary is a group; a key hides payloads alone.
aspace://architecture/sharing · chapter 5

Sharing across teams and organizations

Sharing inside one organization is a question of lanes and admission; sharing across organizations is a question of trust roots. The fabric gives you four independent levers, and the architecture is which of them you pull for which boundary.

LeverWhat it decidesGranularityWhere
Membershipwho is in the group at allpeerthe founding document's policy; INVITE credentials; a POLICY validator
Admissionwho may write or take a given spaceacting agentthe space advertisement; allowlists and credentials
Authorizationwho may vote, take, write, serve a model, rotate a key, issue a directivepeer, or peer/agent with attested agentsMembershipAuthorizer grants, or an OidcAuthorizer with aspace:<op> scopes
Confidentialitywho may read a payloadkey holdergroup content keys, served by key-wrap, rotated on a schedule

Two teams in one organization usually share a group and separate by space and admission: team A's registrars write intake, team B's adjusters take assessment, both read decisions, and the authorizer's VOTE grant names the panel. Two organizations usually get two groups and a bridge: each keeps its own founder, keystore, and content key; the bridge peer is a member of both, admitted on each side under that side's policy, and the records it carries across are the contract between them. Where a partner should see one lane and nothing else, give that lane its own group rather than its own space, since reads in a space are open to the whole group.

A partner that runs peers of its own finds your group by its id and one URL. You publish a beacon's document and the group's signed founding, and the partner joins with join: aspace://<id>, bootstrap.url, and your beacon's PeerId in bootstrap.beacon-peer, so a replaced document is refused. The Spring Boot starter also needs one seed in the group's seeds, such as your beacon's address; the Python and TypeScript bootstrap(url) calls need only the id and the URL. A group started from a text cannot be joined this way, because it has no founding to verify: decide at the start, before the first partner, which kind of group each shared group is. Chapter 2 has the table.

Identity is what makes the levers compose. Every record carries its issuer's signature, so a replica can judge admission and grants without asking anyone. An agent can act under its own certified key (agent-attested) rather than its peer's, which is what lets a grant name one agent on a shared peer and what the AGENT vote granularity counts. Revocation is eventually consistent and follows a freeze rule: what a replica holds stays, what arrives after the revocation is refused, and a key-compromise revocation refuses everything new at once. Plan the revocation path before the first partner joins: who issues it (the founder, or a CA under founder-or-ca), how it reaches every replica (gossip, within a few periods), and what the frozen history means for the records already shared.

The cross-organization blueprintA POLICY group with DID credentials for membership, a group content key with key-wrap so a relay can carry what it cannot read, an ORDERED space for the commitments both sides must see once, and an A2A gateway for partners that run no peer at all. The use-case models on the site sketch it; the pieces each have a chapter in the developer guide.
aspace://architecture/reliability · chapter 6

Building up reliability

A TAKE LEASE IS THE RECOVERY CLOCKwrittentaken, lease 2mworker killedlease lapsesreappearscompleted by a siblingnothing is retried by a scheduler: the lease lapses and the entry is simply available againREDUNDANCY IS A COUNT, THEN A FLOOR3 workers at 99%1-of-N: six nines3 voters at 99%majority-of-N: 3.5 nines5 voters at 99%majority-of-N: 5 ninesafter three or four workers per kind the floor usually matters more than the count:the model provider every worker shares, the one seed everyone dialed, the one peer that holds the ordered log's majority.Diversify the floor before adding agents.
Reliability comes from leases and counts. Work survives a crash because a lease lapses and the entry reappears for any sibling that holds the same binding; there is no retry queue to operate. Availability then follows the arithmetic: one-of-N work needs few workers, majority-of-N decisions need an odd count and a live majority, and past three or four of a kind the shared dependencies usually set the floor.

Reliability on this fabric is built from four mechanisms you compose rather than one you switch on. The lease is recovery: a take holds an entry for a declared time, and a worker that dies lets it lapse, after which the entry is available to any sibling; nothing schedules a retry and nothing has to notice the death. The count is availability: bind the same worker on several peers and the lane drains while any one lives. Replication is durability within the group: an entry lives on every replica from its first gossip period, and a crashed peer's replica is rebuilt by anti-entropy when it returns. And the ordered log is the one place a majority is required, so it is the one place a partition costs availability.

QuestionAnswerKnob
How long until a crashed worker's task is retried?the take lease, plus one ticklease on @SpaceTake: 30 s for a quick task, 2 m for a model call, 10 m for a long job; always longer than the work
How long until a crashed peer is forgotten?the member TTL (30 s by default)SWIM configuration, in code
What if a link drops, or two members never met?a JVM peer retries a lost member's last endpoints and re-dials its seeds when its view is empty, backing off to thirty seconds; the view itself is anti-entropy state, so a member whose introduction was lost is learned from any partner; a Python or TypeScript peer re-dials its seed, re-introduces itself, and re-pulls what it hadnothing to configure; ROBUSTNESS.md has the scenarios
What happens during a partition?each side keeps writing to its replicas and they merge on heal; a lease race may complete the same entry on both sides once; the ordered log's minority side cannot takewhich spaces stand on the ordered log (chapter 7)
What does a reaction do on failure?nothing: a @SpaceNotify that throws is logged and droppedput work that must happen behind a take; put nice-to-have reactions behind a notify
How do I see it?the console: per space the written, completed, queued, and in-progress counts; per peer the suspect state and the channel mode; the wire-traffic panel, frames and bytes by kind as rates with drops by cause and each group's gossip figuresagentspaces.console.*, a read-only peer

The arithmetic for reliability is worth internalizing. Work that any one of N workers can do has availability 1 - (1 - a)^N, so three workers at 99% give six nines, and each further independent worker adds two more. A decision that needs a majority of N has a harsher curve: three voters at 99% give three and a half nines, five give five. Past three or four of a kind the floor usually takes over: the model provider all workers call, the seed every peer dialed, the ordered log's majority set. Diversify the floor (two providers, two seeds, log members on separate hosts) before adding agents.

Three caveats worth noting.

  • There is no universal cutoff at three or four. Every additional independent worker in the any-one case reduces the failure probability by 100 times. Whether that helps materially depends on the shared dependencies' failure rates and your target.
  • Availability and correctness are different. Having a majority online does not establish that an agent's decision is correct. The analogous voting calculation requires independent errors and a decision rule under which a correct majority produces a correct result.
  • Diversity must remove the actual shared failure. Two providers help only if failover works; separate hosts may still share power or networking. A seed used only during initial discovery may limit bootstrap availability without limiting ongoing operation. AWS's reliability guidance says the same.
The honest limitsA lease race can complete the same entry once on each side of a partition; if that is unacceptable, the stage climbs to the ordered log (chapter 7). A reaction that must not be lost is a take. There is no durable workflow history and no saga engine: a saga is explicit request, success, failure, and compensation records, each leased, each a stage (workflow guide, chapter 10).

A note for readers who know work-stealing schedulers. Cilk, Dask, a Kubernetes work queue, and Nomad's clients all pull; a LEASE_RACE take is the same pull with the signed claim as the arbiter and the lease as the acknowledgement, and no scheduler anywhere. Two things that family does and the fabric does not: it never runs a speculative duplicate while a holder lives (the deadline-as-leased-reminder pattern of the workflow guide is the substitute, and a quorum take and a speculative sibling are proposed FUTURE rows), and it never votes a quorum of results on its own (fork, join, and ballot are the substitute).

aspace://architecture/ladder · chapter 7

The consistency ladder

idempotent writea deterministic entry id; a repeat is a no-opcost: freeleaves open: duplicates foldLEASE_RACE takesigned claim, settle window, lowest (hlc, peer) winscost: one settle windowleaves open: a transient duplicate across a partitionAUCTIONclaims carry bids, the lowest price winscost: O(diameter) roundsleaves open: lying bidders; untrusted pricesLEASED ticketa join or reduce key serialized by a leased ticketcost: one claim per keyleaves open: a duplicate is possible under partitionORDERED loga Raft log commits the claim; exactly one completioncost: a quorum round trip per claimleaves open: a minority partition cannot takeclimb one rung at a time, when the workload proves it must; most stages stop at the secondstronger guarantee, higher cost
The consistency ladder. Each rung buys a stronger once-only guarantee at a higher cost and a new failure mode. An idempotent write needs nothing; a lease race needs one settle window; an auction converges in a few rounds and trusts its bidders; a leased ticket serializes a key without a quorum; the ordered log is exactly once at the price of a live majority. Nothing on a lower rung has to move when a stage climbs.

The rungs

Each space promises one thing about who may complete an entry, and the promise has a price. The library's design rule is escalation: a space starts on the lowest rung that is correct for its workload and climbs one rung when the workload proves it must. Nothing on a lower rung has to move when one stage climbs, because the rung is a property of the space.

RungThe promiseThe protocolThe costWhat it leaves open
Idempotent writea repeat changes nothinga deterministic entry id; the CRDT folds the duplicatenonenothing, for stages that can be written this way
LEASE_RACE takeone holder per lease perioda signed claim, a settle window (200 ms by default, shorter than one 250 ms tick, and set per space), the lowest (stamp, peer) winsone settle window of latencya transient duplicate across a partition or a back-dated claim
AUCTIONthe cheapest holderclaims carry a price from a bid function; convergent bidding in a few roundsa few rounds; one bid function per space per peera lying bidder; prices are untrusted
LEASED ticketonce per key, per leasea join or reduce key serialized by a leased ticket taken under the raceone claim per keya duplicate under partition
ORDERED logexactly one completion, fleet-widea Raft log commits each claim; @OrderedTake, ORDERED joins and reducesa quorum round trip per claim; a fixed member set; a live majoritynothing about external systems: make the irreversible call last, keyed by the entry id

When to climb

Climb from the first rung when a stage has a side effect you cannot make idempotent. Climb from the lease race to an auction when workers are not interchangeable and the choice among them is a cost you can compute. Climb to a leased ticket when a stage fires once per key and keys can arrive out of order (a join, a running total). Climb to the ordered log when a duplicate costs money, safety, or trust, and accept the quorum: pick the three or five peers with the steadiest uptime for the member set, host the log on the space the stage completes in, and keep the irreversible external call as the last step, keyed by the entry id, so that a retry after a crash finds it done. The Party Bus keeps one of its nine spaces on the ordered log; example 18 keeps one of five. Most stages stop at the second rung.

The things in betweenTag shards (a partial replica of a large space), filters and tags (a worker that sees only its slice), settle windows (longer where gossip is slower), and admission (fewer writers) are all ways to narrow a lane without climbing a rung. The developer guide's conflict strategies and the workflow guide's exactly-once chapter work through the choices.
aspace://architecture/deciding · chapter 8

Deciding: votes, quorums, estimates

A VOTE: QUORUMProposalid, question, options, quorumMotion or @ProposeBallotsigned, one per voterBallotsigned, one per voterBallotsigned, one per votertally at quorumDecisionwinner, tally, granularityauditable, recomputable from the space; one peer or one agent is one voterevoke a voter and the count reopensAN ESTIMATE: PUSH-SUMpeervalue 10peervalue 30peervalue 20halves exchanged each roundestimate 20.0settled: unchanged for N roundsapproximate, fast, no electorate; shares go only tomembers that advertise the aggregate; a silent nodenever settles on its own valueuse for sensing; never for a trust-sensitive decision
Two ways to decide, for two kinds of question. A QUORUM vote is a space full of signed ballots: slow, auditable, and the right tool when the answer must be defensible. A push-sum estimate is a converging number every member ends up holding: fast, approximate, and the right tool for sensing load, counts, and thresholds. The ladder's rule applies here too: an estimate that gates money or safety becomes a vote.

A fleet decides in two ways, and the architecture is which one each decision gets. A vote is a space of signed ballots: a proposal names its options and quorum, each eligible voter writes one ballot, and the decision closes when the tally reaches the quorum, recomputable by any replica from the space itself. An estimate is a push-sum aggregate: each member contributes a number, members exchange halves of their mass every gossip round, and every member converges on the fleet-wide sum, average, count, extremum, or quantile. The vote is slow, auditable, and bounded by an electorate; the estimate is fast, approximate, and open.

Use a vote whenUse an estimate whenUse a workflow join when
the answer must be defensible laterthe answer is a number that a few percent of error cannot hurtthe answer is the presence of several records about one case
the electorate matters (grants, agent-attested voters, one vote each)every member's contribution counts equally and nobody can be excludedthe parts arrive from different stages and the fan-in fires once per key
money, safety, or policy turns on itload, backlog, counts, thresholds, readinessthe stage is data-driven rather than opinion-driven

Three decisions shape a vote. Granularity: one peer is one vote by default; name peer/agent in the vote grant and the count becomes one attested agent one vote, which is what lets a council of several agents sit on one peer. Quorum: too low and one peer decides, too high and a lost peer stalls the vote until its lease lapses; three of five is the usual answer, and an odd electorate avoids ties (a tie closes to the lexicographically first option, so avoid relying on it). Who opens it: a stage returns a Motion when its question, options, or quorum depend on the cue, or carries @Propose when the shape is fixed and only the key varies; either way the proposal opens once per id however many replicas see the cue.

Two decisions shape an estimate. What settles it: a settle rule fires when the estimate has held for N steps within a tolerance, a step being a gossip round the node took part in after it first heard the epoch, or the epoch going quiet, so a node nobody pushes to never settles on its own value and a plateau before a late contribution is not settled; pick N from the fleet's diameter and the spread of the contributions and the tolerance from what the consumer can tolerate. Rounds run about once a second under a driven peer whatever the 250 ms tick, so forty steps is about forty seconds. Example 18 uses settleTicks = 40, since one reserve follows the ordered log. Who takes part: shares go only to members whose aggregate advertisement is live, so a console or a client that has not started the capability never swallows mass; a member that must count, must advertise. An estimate does not detect a poisoned contribution; the moment one would matter, the decision is a vote.

MAJORITY_GOSSIPThe vote capability's gossip mode decides by push-sum over preferences: fast, approximate, no ballots to audit. It is the right tool for a leader hint or a soft threshold and the wrong tool for anything the vote's audit trail exists for. The annotations bind QUORUM votes; the gossip mode is reached through the capability's helpers.
aspace://architecture/workflows · chapter 9

When a workflow, and how many spaces

A workflow on this fabric is nothing more than records that carry a case id moving through spaces, each stage a binding that takes or reacts and writes the next record. That is why there is no engine: the vocabulary of the workflow guide (sequence, map, fork, fan-out, join, gather, reduce, compose, filter, route, branch, decide, exactly once, deadline, loop, compensation, the human step, the model step, the planner) is each a way of binding, and the engine's state is the spaces themselves. Reach for it when a business case passes through several hands and each hand must survive a crash; keep a single take when one worker does the whole job.

ShapeWhenWhat to watch
One space, one record type per stagea short pipeline whose stages are all takes and all workers read the laneevery stage's lease fits its work; the lane's strategy fits every stage
Several spaces, one groupstages differ in strategy, admission, or retention; a join gathers parts from several lanes; one stage needs the ordered loga part may live in another space; an ORDERED join lives on the space that hosts the log; reactions fire on every replica, so what must happen once is a take, a fleet-wide join, a vote, or an idempotent write
Several groupsthe parties differ: a tenant's flow hands a commitment to a partner's, a conversation group reports to a mission groupa bridge peer in both groups, bound by group name; the bridged record is the contract; no shared ordered log across groups, so the bridge takes once on one side and writes idempotently on the other

Deadlines are leased reminders: a stage writes a reminder with the lease it is willing to wait, and a reaction on the lease's expiry escalates; the reaction fires within a tick of the lapse and nowhere else. Loops are stages that write their own cue again with a bounded counter. Compensation is a stage like any other, written when a later stage fails, taken by the agent that can undo. A planner or a model is a stage whose implementation is a model call; the take's lease is renewed on every model round-trip by the host libraries, so a long think does not lapse the task.

Where it is worked throughExample 18 is a nine-stage claims flow over five spaces and three peers, with one stage per concept and a test that reads every property from a space; the workflow guide walks it chapter by chapter. The four flagship applications (petclinic, code, audit, intake) are complete flows with their own guides.
aspace://architecture/recipes · chapter 10

Deployment recipes

Edge fleet3 to 15 listening peers on onenetwork; every peer a seed;LEASE_RACE spaces; mtlsHub and fielda few listening seeds, manydial-only peers; relay forpeer-to-peer; the hub holds theordered logMulti-sitea rendezvous per site, relaysacross; one group; ORDERED onlywhere a majority stays livePartner integrationa second group; a bridge peer inboth; content keys; A2A or MCP atthe edgePolyglot workersJava coordinator with Python andTypeScript workers dialing in onplain TCP (dev-local) or fronted byTLSHuman in the loopa console peer withcommand-and-control grants; votesfor approvals; deadlines as leasedreminders
Six of the deployment recipes. The table below lists all of them, including the shapes that chapter 11 describes. Each is a composition of the patterns in this guide: a topology, a set of boundaries, a rung on the ladder for each space, a way to decide, and an edge for the outside world. The chapters above say when each piece earns its cost.
RecipeTopology (ch. 1 to 3)Boundaries (ch. 4, 5)Guarantees (ch. 6 to 9)Edge
Edge fleet3 to 15 listening JVM peers on one network, every peer a seed, mtlsone group, one space per lanelease race everywhere; three workers per kind; a vote where a human would have signeda console peer
Hub and fielda few listening seeds in the hub, many dial-only field peers (laptops, boxes, Python and TypeScript clients), a relay for field-to-fieldone group; field lanes admitted by allowlistthe hub holds the ordered log; field work is lease-race takes with long leasesa console and, for partners, an A2A gateway in the hub
Multi-sitea rendezvous per site, relays across, one groupspaces per lane; a content key if the sites' links are not trustedordered log members on the site with the steadiest links; settle windows sized to the WANa gateway per site if partners dial in
Enterprise discovery (ch. 2)a beacon per site serving the document behind TLS; its records in the corporate DNS, or only the pointer record; every agent configured with bootstrap.order: [dns-sd]one group per mission; the beacon's keystore kept; partners pin the beacon's PeerIdunchanged: discovery only answers where to dialthe DNS records the beacon prints
Partner integrationthe partner runs peers of its own or dials an A2A gatewaytwo groups and a bridge peer; the partner group under POLICY or INVITE; a content key per groupcommitments on an ORDERED space in the group that owns them; idempotent writes across the bridgeA2A for agent protocols, MCP for model tooling
Polyglot workersa JVM coordinator with Python and TypeScript workers dialing inone group; the workers' lanes admitted by allowlist; dev-local or a TCP-fronting peerclients take under the lease race and contribute to estimates and votes; once-only stages take through the ordered log, whose voters sit on JVM peersthe trilingual demo's council is the template
Human in the loopany of the above plus a console peer with the directive grantan audit space for every commandapprovals as votes with the panel in the grant; deadlines as leased reminders that escalatethe console's command-and-control: dispatch, cancel, pause, resume, drain
Browser edge (ch. 11)a hub on wss behind the ingress and https for the fallback, browsers as visitors or residents, Workers cells as assets, WebRTC between browsersthe hub's origin allowlist; a group per applicationno browser votes; the hub holds the ordered desk; a cell answers a retry from memorythe hub is the relay and the gateway
Tactical (ch. 11)dtls beside tls, the sealed multicast beacon on the segment, a relay at the posta content key; one group per missionleases sized to the outage; the ordered log at the post only; the minority side stops takinga console at the post
LLM resource pool (ch. 11)many small workers with no API key, three model servers holding the credentialsa models space under AUCTION; the model-serve granttoken-priced bids; leases renewed by progress; usage sensed by push-sumthe usage panel
Coalition (ch. 11)a group per partner and a mission group, a bridge peer per partner, transports as tacticalINVITE membership and a content key per groupQUORUM votes with signed ballots; the space as the audit traila CoT bridge (not yet built)
Volunteer compute (ch. 11)heterogeneous workers under AUCTION, one founder-issued join credential per volunteerone group, a work lanetwo copies per task, joined and voted; bulk over the block exchangenone
Local-first (ch. 11)an automerge-repo or Yjs application over the store adapters, through a wss hubthe group as the perimeterthe library's CRDT; delivery among members by the fleetnone
aspace://architecture/architectures · chapter 11

Architectures a line of configuration selects

The 0.4.0 transports and hosts (ws, wss, http, https, webrtc, DTLS over UDP, the sealed multicast beacon, the bootstrap family, browsers and Workers cells as members) widen what a fleet can be without widening what an agent has to know. Each architecture below is a shape the fabric already supports; what selects it is a few keys of the one agentspaces: configuration every host reads. The entries are ranked by how much one line changes, each names the mechanisms that carry it, its failure model, and the system a reader may know it from, so that someone with a Tailscale, BOINC, TAK, or Syncthing background recognises the shape at once. reviews/p2p-architectures-research.md holds the survey these entries rest on: thirty-five systems mapped onto the fabric's mechanisms, with the verdicts fits as built, fits with configuration, needs a capability, and out of scope.

RankArchitectureWhat one line changesResembles
1Network-bridging edgea wss block makes a browser tab a memberDERP on 443, PeerJS, a WebRTC SFU for the relay
2Tacticala dtls block and multicast.seal put the fleet on a lossy segmentDDIL edge computing, DTN store-carry-forward, Reticulum announces
3Low-level agents sharing LLM resourcesstrategy: AUCTION and the model-serve grant remove every worker's API keyCilk and Dask work stealing, a Kubernetes work queue
4Centralized core, distributed edgeroles and an ordered log on three peers make the rest dial-onlyNomad servers and clients, Syncthing's discovery server
5Network-bridging enterprisebootstrap.domain: auto and require-attestation join sites through the DNS and CA already thereTailscale with corporate DNS, Matrix federation
6Tactical/militarymembership: invite and a content key per partner draw the coalitionTAK Server federation, DDIL doctrine, SSB's signed feeds
7Highly distributedmulticast.enabled, or one seeds entry when the members are apart, nothing elseSecure Scuttlebutt, Syncthing, Trystero rooms
8High-level business agentsan ordered log behind one space and @OrderedTake on its stageTemporal's workflow with the space as the state
9Federated multi-organizationa second groups[] entry on a bridge peerMatrix homeservers, TAK federation
10Volunteer compute and work stealing by auctionAUCTION plus a fork-join-vote workflowBOINC's replicated validation, MapReduce backup tasks
11Local-first over the store adaptersone constructor argumentSyncthing for sync and relay, Hypercore without the DHT

1. Network-bridging edge: browsers and phones through a hub, WebRTC between them, Workers cells as assets

The picture. A Java or Node hub listens on wss behind the ingress on 443, and on https for the long-poll fallback. Browser tabs join as visitors or residents through the hub, take and watch like any peer, and open data channels to each other signalled through the hub's pipes. A celld or Cloudflare deployment publishes its Durable Objects, D1 tables, and queues as AssetCards that Java agents query and invoke. The hub is also the relay for browsers that cannot reach each other and the service host for any ordered desk.

Mechanisms. agentspaces-transport-http with its origin allowlist and in-band session attestation; the portable TypeScript core and the residency table; @agentspaces/browser for a tab that outlives its page; @agentspaces/workers with aspace:cap/asset-invoke; aspace:cap/rtc-signal with attestation by signed DTLS fingerprint; the relay budget.

Failure model. A frozen tab lets its leases lapse and its work reappears; two tabs of one origin are one member under a Web Lock; a cell that moves mid-invocation answers a retry under the same invocation id from memory; the hub's loss cuts browsers off until they re-dial, so run two hubs behind the ingress. No browser is a voter.

Configuration. Hub: agentspaces.transport.wss: { enabled: true, bind: 0.0.0.0:7503, advertise: fleet.example.com:443/aspace, allowed-origins: [https://app.example.com] }, agentspaces.transport.https: { enabled: true, bind: 0.0.0.0:7503, advertise: fleet.example.com:443/aspace } (the same bind as wss, which an https block may share; a block with no bind only dials), agentspaces.transport.webrtc.enabled: true, agentspaces.roles: [RELAY]. Page: connect({ group, seeds, worker }) from @agentspaces/browser, or a Peer from @agentspaces/client. Cell: the wrangler.json manifest and the connector cell from @agentspaces/workers.

2. Tactical: intermittent links, DTLS over UDP, sealed multicast, relays, disconnected operation

The picture. Vehicles, kits, and a command post on a radio or satellite segment that loses five to thirty percent of packets and drops for minutes. Peers listen on dtls beside tls; the multicast beacon finds neighbours on the segment with its datagrams sealed under the group's content key; a relay at the post joins kits that cannot dial each other; each segment keeps writing, taking, and completing on its replica and reconverges on heal.

Mechanisms. The dtls binding with Connection ID and the recovery sublayer; Transport.capabilities(); agentspaces.multicast.seal; the RELAY role; delta-CRDT replication; leases sized to the outage.

Failure model. An association silent for 60 s fails and SWIM marks the peer suspect; the lost-member retry re-dials by priority and a fresh handshake attests anew; a lease race may complete the same entry once on each side of a partition, so irreversible stages sit on an ordered log at the post, which stops taking on the minority side. DTLS 1.2 shows peer identities to a passive observer; use quic or tls where that matters. The courier and air-gap case (a shared-filesystem transport) is specified and unshipped.

Configuration. agentspaces.bind: 0.0.0.0:7500 with agentspaces.transport.tls.enabled: true (the tls endpoint, which the peer advertises first, at priority 0), agentspaces.transport.dtls: { enabled: true, bind: 0.0.0.0:7502 } (which the peer advertises next, at priority 1 unless the block sets priority), agentspaces.multicast: { enabled: true, seal: true }, agentspaces.groups[].content-key, agentspaces.roles: [RELAY] at the post, take leases of minutes on field stages, and the ordered log's member set on the post's peers only, wired in code (the core and edge shape says how).

3. Fleets of low-level agents sharing LLM resources

The picture. Twenty to two hundred worker peers, each a small @SpaceTake class with no provider module and no API key, write model requests; three model-server peers hold the provider credentials, bid by token price under AUCTION, stream chunked deltas, and renew the worker's lease by progress; a push-sum epoch sums tokens per model and a supervisor sheds load when the estimate crosses a threshold.

Mechanisms. FleetChatModel and ModelServer (Spring AI), their LangChain4j and Micronaut twins over agentspaces-model-wire, AUCTION with @BidFunction and token-priced bids, the MODEL_SERVE grant, TakeLeaseAdvisor, usage sinks into push-sum epochs, @OnEstimate.

Failure model. A dead server's request is taken by another with the next attempt number; a dead worker's task reappears after its lease; a lying bidder is an accepted risk, so price bids from a measured rate and keep the server set under the grant; the usage estimate is approximate and the usage audit entries are for billing. This is work stealing without a scheduler: the take is the pull, the signed claim is the arbiter, the lease is the acknowledgement; what it does not do is run a speculative duplicate while a holder lives (the deadline-as-leased-reminder pattern substitutes) or vote a quorum of results (fork, join, and ballot substitute).

Configuration. Model space: agentspaces.groups[].spaces[]: { name: model-requests, strategy: AUCTION }, the name the Spring AI client and server use unless model-client.space and model-server.space change it. Servers: agentspaces.springai.model-server: { enabled: true, models: [...], routing: price, prices: { <model>: <price per thousand tokens> } } (price bids need both routing: price and prices; the server refuses to start when its space does not run AUCTION) and agentspaces.security.grants.model-serve: [<server PeerIds>], the provider key only there. Workers: agentspaces.springai.model-client.enabled: true and no provider module. Usage: agentspaces.springai.usage.enabled: true. Scale is another instance of the same worker class.

4. Centralized core with a distributed edge

The picture. A core of three to five listening JVM peers on steady hosts holds the ordered logs, the rendezvous cache, the relay, the console, and the gateways; the edge is as many dial-only peers as the business has laptops, boxes, Python scripts, and browsers, each taking from lanes admitted by allowlist with long leases. Sites reach the core through DNS; browsers through wss.

Failure model. Losing an edge peer costs one lease; losing the rendezvous slows discovery and changes no result; losing a core majority stops the ordered stages and nothing else. The floor is the shared provider and the seed list: give every edge peer two seeds.

Configuration. Core: agentspaces.roles: [RENDEZVOUS, RELAY] on each core peer, a beacon from the agentspaces-beacon project beside them, and the payments log's member set (core-a, core-b, core-c) wired in code with OrderedTakes and group.ordered("payments", coordinator). The starter's agentspaces.capabilities.ordered-log cannot infer that set and refuses to start without a @ProvidesCapability bean that carries it, and a TypeScript Node service host reads an ordered-logs: [{ space, members, log-dir }] block from its fleet file. Edge: agentspaces.groups[].seeds: [tls://seeds.corp.example:7500] (every address of the name is dialed), no bind, spaces[].admission: ALLOWLIST.

5. Network-bridging enterprise: sites joined through DNS-found relays, TLS passthrough, zero-trust attestation

The picture. Three sites, each behind its own firewall and ingress. One peer per site carries RENDEZVOUS and RELAY, publishes the DNS-SD records its beacon prints under the corporate domain, and listens on tls directly and wss through the ingress in passthrough mode. Every peer runs require-attestation under the enterprise CA with CRLs; a certificate revocation ejects a member at its next handshake fleet-wide. A relay that forwards what it cannot read, under a byte budget, is the DERP or TURN of this shape; a beacon that holds no secret and admits nobody is its coordination server.

Failure model. A poisoned DNS answer delays a join and substitutes nothing when the group id is configured; a terminating proxy breaks attestation and the node refuses the link rather than downgrading; relayed frames are refused under required attestation except the WebRTC signal, so dial-only peers at two sites need a common relay both sites attest.

Configuration. agentspaces.bootstrap.domain: auto, agentspaces.groups[].join: aspace://<id> for a group with a signed founding, with at least one entry in that group's seeds because the starter does not count the locators built from agentspaces.bootstrap.* toward a join: group and refuses to start with no seed, no BeaconLocator bean, and no multicast (a group started from a text takes the same founding: on every peer and needs no seed for this), agentspaces.bootstrap.accept-group-id-from: [url], agentspaces.transport.tls.require-attestation: true with the trust store and CRL, agentspaces.transport.wss.advertise: site-a.corp.example:443/aspace behind a passthrough ingress, agentspaces.roles: [RENDEZVOUS, RELAY] on one peer per site, agentspaces.security.profile: zero-trust where the relays themselves are distrusted, with the agentspaces.security.oidc keys issuer, audience, and jwks-url that profile requires (it refuses to start without them, and the identity provider's scopes replace security.grants).

6. Tactical/military: DDIL, coalition perimeters, TAK-style situational awareness, auditable decisions

The picture. A coalition operation runs one group per partner and one mission group all partners join; each partner admits its own members under its own founder and join credentials and holds its own content key; the mission group carries position and status records as leased entries that go stale by lease, as a Cursor-on-Target event goes stale by its stale attribute; a bridge peer per partner converts CoT to entries and back; decisions that commit a unit are QUORUM votes with signed, recomputable ballots.

Mechanisms. Everything in the tactical shape, plus groups as perimeters with INVITE membership and founder-issued or delegated join credentials, content keys with key-wrap so a partner's relay carries the mission group it cannot read, a bridge peer in two groups, @Propose and @Ballot with the VOTE grant naming the panel, the space as the audit trail. The CoT bridge is a connector in the shape of agentspaces-connect-core that the tree does not hold yet.

Failure model. A partner's revocation of its delegation refuses every credential issued under it in one gossip round; a key-compromise revocation refuses everything new at once; a vote with a lost voter stalls until the voter's lease lapses, so size the quorum to the members that stay reachable; types, tags, and issuers are visible to the relay, so a sensitive record type gets its own group.

Configuration. Per partner: agentspaces.groups[]: { name: partner-a, membership: invite, founder: <founder PeerId>, join-credential: <credential>, content-key: ... } on each member, with founder and join-credential unset on the founder itself; the mission group the same way with the mission's own founder and content key; agentspaces.security.grants.vote: [<panel PeerIds>]; the bridge peer lists both groups; transports as in the tactical shape.

7. Highly distributed: no core at all

The picture. A lab floor, a workshop, a convoy, or a group of friends: every peer listens, any member is a seed, nobody runs a rendezvous, and discovery is the multicast beacon on a segment, or one seed (a member's address, or a beacon from the agentspaces-beacon project) when the members are apart. Every lane is LEASE_RACE or AUCTION; nothing needs a majority, so no ordered log and no peer whose loss matters.

Failure model. Any peer's loss costs its leases; a partition yields two working halves that merge on heal with at most one duplicate completion per entry per side; a group with fewer members than the fan-out prunes nothing and settles claims in a few ticks. A beacon, if used, is the one shared piece, and losing it slows the next newcomer and nothing else.

Configuration. agentspaces.groups[]: { name: floor, founding: "<string>" }, agentspaces.multicast.enabled: true (and seal: true with a content key), agentspaces.bind: 0.0.0.0:7500, no roles, no ordered log; or, apart from each other, agentspaces.groups[].seeds naming one member or a beacon on every peer after the first.

8. Fleets of high-level business agents: planner-driven, few peers, ordered desks and votes

The picture. Three to five JVM peers host a planner (Embabel, or a Spring AI or LangChain4j tool loop over the fleet's live cards), a handful of specialists, an ordered desk for the stage that must happen once, a council that votes on what a human would have signed, and a console with the directive grant. Each business case is a record carrying a case id through five or so spaces; a planner sees a Workers operation (reserveStock on workers://inventory/do/Inventory) beside the agent actions.

Failure model. A crashed planner's take reappears with its chat memory checkpoint on another peer; the ordered desk stops on a minority partition and the rest of the flow continues; the irreversible call is last and keyed by the entry id.

Configuration. agentspaces.groups[].spaces[] with strategy: LEASE_RACE for intake, AUCTION for assessment, and the default for payments, whose stage takes through the ordered log with @OrderedTake (a space strategy cannot be ORDERED); the payments log's members a, b, and c wired in code as in the core and edge shape; agentspaces.security.grants.vote: [...]; agentspaces.console.enabled: true with command.enabled: true; agentspaces.embabel.remote-actions-enabled: true or the Spring AI fleet tools.

9. Federated multi-organization fleet

The picture. Two or more organizations each run their own group with their own founder, keystore, authorizer, and content key; a bridge peer per pair is a member of both and carries the contract records across, taking once on one side and writing idempotently on the other; partners that run no peer dial an A2A gateway; membership is INVITE under a delegation or POLICY under credentials.

Failure model. No shared ordered log across groups, so the bridge prevents a cross-group duplicate by an idempotent write; revocation is per group and eventually consistent under the freeze rule; the bridge is the one peer both sides must trust.

Configuration. On the bridge: two agentspaces.groups[] entries, each with its own join, one seed, bootstrap.url (the partner's beacon document, bootstrap.beacon-peer pinned), the join-credential that group's INVITE policy asks for, and content-key (the membership policy comes from the group's signed founding); an agent bound to a group on each side; agentspaces.a2a.enabled: true on the gateway.

10. Volunteer compute and work stealing by auction

The picture. A pool of heterogeneous workers (a lab's workstations, a partner's spare GPUs, Python scripts on laptops) take long tasks under an AUCTION whose bid function prices locality, queue depth, and hardware; each task forks into two copies whose results a join gathers and a ballot reconciles, BOINC-style; inputs and outputs over 64 KiB ride the block exchange.

Failure model. A vanished volunteer lets its lease lapse; a lying bidder takes work it never performs and the lease-lapse re-auction is the mitigation; a wrong result is outvoted only when the two copies ran on different peers, which the bid function must enforce by excluding the first winner.

Configuration. agentspaces.groups[].spaces[]: { name: work, strategy: AUCTION }, membership: invite with one founder-issued join-credential per volunteer (and founder naming the issuer on each volunteer), a @BidFunction per worker kind; the fork-join-vote shape from the workflow guide.

11. Local-first applications over the store adapters

The picture. A collaborative application on automerge-repo or Yjs changes one constructor argument and its sync travels as signed frames between fleet members, through the hub, the relay, or a data channel, under the group's content key; the document model and its conflict resolution stay the library's, and the fleet supplies membership, transport, and the perimeter.

Failure model. The CRDT converges on the library's terms; the fleet guarantees delivery among members and the perimeter; pipes give up the space's retries and audit trail, so a document that must be auditable is written as entries instead.

Configuration. new Repo({ network: [new AgentSpacesNetworkAdapter(pipes)] }) or new AgentSpacesProvider(pipes, "room", doc); a wss hub as in the first shape, or the multicast beacon on one segment.

What the survey asks for nextEight rows the mapping proposes, none built: a CoT bridge connector; a quorum take and a speculative sibling on @SpaceTake; a local-health multiplier on suspicion for hosts with drifting timers; partial membership views for fleets past the full-view budget; datagram pipes and lanes before DDS-class telemetry; the shared-filesystem transport for couriers and air gaps; the Zenoh and DDS bridges. reviews/p2p-architectures-research.md §6.
aspace://architecture/checklist · chapter 12

The architect's checklist

  • Peers. Each peer's purpose is one of worker host, coordinator, rendezvous, relay, console, gateway, or bridge, and the count of each follows chapter 1's table. Every worker kind has three or four instances on different peers.
  • Clock. The tick and gossip period are chosen, documented, and the same for every member; every lease, settle window, and deadline is a stated multiple of them.
  • Bootstrap. Every peer has at least two seeds; each site has a rendezvous; dial-only peers that must reach each other have a relay. The founding path (starter string, or a kept founder key in code) is decided with the revocation and rotation paths it implies. Each group's chain fits its network (the table in chapter 2), a beacon has a keystore and TLS in front and its records are registered, and a partner joins by id with the beacon's PeerId pinned.
  • Profiles. Every peer runs a security profile that meets the others'; where a non-JVM client dials in, the TCP listener it reaches is deliberate and require-attestation is off only where that is intended.
  • Lanes. Each space has one strategy, one admission rule, one retention, and a reason for existing; records carry the case id; references are ids.
  • Perimeters. Each group has a membership policy, a content-key decision, and an authorizer with named grants; any bridge peer is listed with the records it carries across.
  • Rungs. Each space's rung on the ladder is stated with the workload fact that justifies it; ordered logs have a named member set of three or five and the irreversible call is last.
  • Decisions. Each decision is a vote or an estimate with the reason; votes have a quorum, a granularity, and an electorate; estimates have a settle rule and only advertising participants.
  • Failure. For each stage: what lapses, who picks it up, what a reaction drops; for each partition: which side keeps working and which stops.
  • Observation. A console peer exists; the operator can name the written, completed, queued, and in-progress counts per space and the suspect state per peer; every command is audited.
  • Proof. The flow has a test that reads every property from a space, runs under a crash, and (where languages mix) runs the workers in their own processes.
Where to go nextThe developer guide for each API and property named here, the workflow guide for the stage patterns and example 18, RELIABILITY.md for the availability arithmetic, and SPEC.md and TECH-SPEC.md for the formats and the rationale.