Perl JSON API Clients: Why I Built a Transport-Independent Foundation

This article explains why a transport-independent foundation for Perl JSON API clients is a practical design choice: it lets the same request-building, validation, and response-handling code work over HTTP, local files, or in-memory test doubles, without rewriting the client for each transport. You enter the API endpoint and payload once, and the foundation handles serialization, error mapping, and retry logic in a way that is decoupled from how the bytes actually travel.

The Problem with Transport-Coupled Clients

Most Perl JSON API clients are written with HTTP baked in from the start. They take a URL, use something like LWP::UserAgent or Mojo::UserAgent to send the request, and then parse the JSON response. That works fine for the common case, but it creates hidden coupling: the client logic is tangled with network specifics such as timeouts, proxies, TLS settings, and connection reuse. If you later need to read from a file, replay a recorded response, or simulate a server in a test, you are stuck either mocking the HTTP layer or duplicating the client code.

The coupling also makes unit testing harder. Every test that calls a method on the client triggers a network call unless you have a mock HTTP backend installed. A transport-independent foundation flips this: the client core only knows about requests and responses as data structures. The transport is an injected role or callback, so tests can use a trivial in-memory transport that returns canned JSON.

What a Transport-Independent Foundation Actually Looks Like

In Perl, the typical design uses a role (e.g., with Moo or Moose) that defines the JSON handling and request/response objects, but leaves the actual byte-sending to a method that a consuming class provides. For example:

package MyAPI::Client;
use Moo;
with 'MyAPI::Role::JSONTransport';

sub request {
    my ($self, $req) = @_;
    # $req is a hashref with method, path, params, body
    my $http = $self->http_ua;   # injected
    my $resp = $http->request($self->build_http_request($req));
    return $self->parse_response($resp);
}

The role provides build_http_request and parse_response which handle JSON encoding/decoding, header setting, and error detection. The actual sending is delegated to whatever object is set as http_ua. That object could be a real user agent, a file reader, or a mock. The key is that the client code never calls LWP::UserAgent->new directly; it receives the transport as a dependency.

Why Not Just Use an Existing HTTP Client?

Perl has excellent HTTP clients: LWP::UserAgent, Mojo::UserAgent, HTTP::Tiny, and Furl, to name a few. Each has its strengths. LWP is mature and widely used; Mojo has a clean event-loop integration; HTTP::Tiny is core and lightweight. But they all share one assumption: you are talking to a remote server over a socket. When you need to do any of the following, the HTTP client becomes a liability:

• Read a response from a file (e.g., a cached API response)
• Simulate a server in tests without starting a local socket
• Support multiple protocols (HTTP, HTTPS, maybe a Unix socket) without changing client logic
• Add a custom retry or caching layer that sits between the client and the network

A transport-independent foundation lets you swap the transport without touching the client code. This is not a new idea—it is the same principle behind PSR-7 in PHP or the Adapter pattern in general—but it is often overlooked in Perl because the standard approach is to just use LWP and be done.

How to Choose the Right Transport for Your Use Case

When you build a transport-independent client, you still need to pick a concrete transport for production. The choice depends on your environment:

• HTTP::Tiny – core module, no dependencies, good for simple scripts and minimal installs.
• LWP::UserAgent – battle-tested, supports cookies, proxies, and custom headers easily. Good for complex HTTP interactions.
• Mojo::UserAgent – if you are already using Mojolicious, it integrates well and supports non-blocking.
• Furl – high-performance, but requires XS compilation.

For testing, you can inject a simple object that just returns a fixed response. For example:

package TestTransport;
sub request { return { status => 200, content => '{"ok":true}' } }

This avoids the need for Test::MockModule or a fake HTTP server. The client code is tested in isolation, and the transport is tested separately.

Common Pitfalls and Misconceptions

One misconception is that transport independence means you must write your own HTTP client. It does not. It only means that the client core does not assume a specific transport. You still use a battle-tested HTTP client for the actual network I/O.

A common pitfall is over-abstracting. If you only ever use HTTP, adding a transport layer can be over-engineering. The YAGNI principle applies. The foundation is worth it when you have multiple transports or when testing is a significant pain point.

Another pitfall is not handling JSON errors consistently. A transport-independent foundation should centralize JSON decoding and error reporting. If the response is not valid JSON, the client should throw a consistent exception, regardless of whether the response came from HTTP or a file. This requires the transport to return the raw content and the client to decode it, not the other way around.

How It Affects Testing and Maintenance

The biggest win is in testing. With a transport-independent client, you can write unit tests that run in milliseconds, without network access. You can test error handling by having the transport return a 500 or malformed JSON. You can test retry logic by having the transport fail twice then succeed. All of this is possible with an HTTP mock, but the transport-independent approach makes it explicit and avoids the complexity of mocking at the HTTP layer.

Maintenance also improves. When the API changes its response format, you update the parsing logic in one place, not in every test or in multiple transport-specific code paths. When you need to add a new transport (say, a command-line tool that reads from stdin), you write one small class that implements the transport interface.

Alternatives to a Custom Foundation

If you do not want to build your own foundation, there are existing modules that provide some of this functionality. For example, Web::API (though not very popular) attempts to provide a generic API client. Net::HTTP::API is another. However, these are not widely adopted, and many developers prefer to roll their own with Moo roles because it is simple and fits their specific needs.

Another alternative is to use a mocking library like Test::MockModule to mock the HTTP client directly. This works, but it couples your tests to the HTTP client’s interface. If you later change the HTTP client, your tests break even though the client logic did not change.

Frequently Asked Questions

Does this mean I have to write my own HTTP client?

No. You still use an existing HTTP client for the actual network I/O. The transport-independent foundation only means your client code does not directly depend on a specific HTTP client. You inject the transport as a dependency, so you can swap it out for testing or for different protocols.

Is this over-engineering for a simple API client?

It can be. If your API client only ever talks to one HTTP endpoint and you are not worried about testing, then a simple LWP-based client is fine. The foundation becomes valuable when you have multiple transports, complex error handling, or a need for fast unit tests.

How do I handle authentication in a transport-independent way?

Authentication headers are just part of the request. The foundation should build the request including headers, and the transport should send them. For example, an OAuth token can be added to the request in the client core, and the transport just sends the headers. This keeps authentication logic in one place.

What about file uploads or binary data?

JSON APIs typically use base64 encoding for binary data, so the transport only deals with text. If you need multipart/form-data, that is a transport-specific feature. In that case, you might need to extend the transport interface to handle binary payloads, but the core JSON handling remains unchanged.

Can I use this with an event-loop framework like AnyEvent?

Yes, if the transport implements a non-blocking interface. The foundation does not care whether the transport is blocking or not; it just calls the transport’s request method and expects a response. You could have a transport that uses AnyEvent to make async HTTP calls and then returns the response via a callback. The client core would need to support callbacks as well, which is a separate design consideration.

Is this a common pattern in Perl?

Not as common as in other languages, but it is gaining traction. Many modern Perl modules use dependency injection and roles, and the transport-independent pattern fits naturally with that. However, many developers still write simple clients with LWP directly, so this is not yet a mainstream approach.

Related

• Sanity Context Agent Design: Travel Allowance Assistant
• Rough Estimate to Hard Requirement: When?
• Quantum Gravity Explained via Quantum Information