Skip to main content

Extension

Custom Carrier

The question we are most asked is: How to add a custom carrier?

We have experimented and studied approximately ~100 shipping carriers API/web services to design the karrio structure as it is. The good news is that making it easy to add a custom carrier fits perfectly with karrio' vision.

caution

To integrate a custom carrier at this stage, you will need:

  • A minimal knowledge of Python
  • Write code to extend the abstract classes
  • Use the plugin/extension structure available
  • Read and Understand the carrier' API documentation

Note that we are still iterating on the SDK to make it more user-friendly and make it easy to add custom carriers. Therefore, we recommend taking inspiration from the existing carrier integrations. As we discover patterns and things that can be abstracted, we will continue to improve the implementation process.


Steps to integrate a custom carrier

  1. Create a karrio extension package

  2. Generate Python data types

  3. Define carrier connection settings

  4. Set up API calls under proxy

  5. Implement the Mapper functions

  6. Add a database model for karrio server

  7. Add the carrier configuration to the dashboard

info

The package naming convention for extensions is karrio.[carrier_name]


Prerequisites

Make sure that you have set up your development environment as instructed in the development guide.

1. Create a karrio extension package

Once you have karrio installed for development on your machine, You can run the following command to scafold a karrio extension for you carrier.

Terminal
./bin/cli add-extension
Terminal
Carrier slug: freight_express            # Karrio unique carrier_name
Display name: Freight Express # Carrier label that will be used throughout the app
Features [tracking, rating, shipping]: # The carrier features you wish to integrate
Version [2024.3]: # The extension initial version
Is XML API? [y/N]: # Specify whether the carrier' API data format is XML. (n - for JSON APIs)
Generate new carrier: "Freight Express" extension with id "freight_express" and features [tracking, rating, shipping] [y/N]: y

This command will create a karrio compatible extension folder under modules/connectors/[carrier_name] with all the boilerplate code required for your integration. From there, all you have to do is to implement the API data mapping from and to karrio unified interface. And add API contract tests to ensure that karrio generates the right requests for your API.

Extension anatomy

Considering the vision we aimed to achieve with karrio, the codebase has been modularized with a clear separation of concerns to decouple the carrier integration from the interface abstraction. Additionally, each carrier integration is done in an isolated self-contained Python package.

As a result, we have a very modular ecosystem where one can only select the carrier integrations of interest without carrying the whole codebase.

Most importantly, this flexibility allows the integration of additional carrier services under the karrio umbrella.

info

karrio makes shipping API integration easy for a single carrier and in a multi-carrier scenario, the value is exponential.

Signature

The Mapper is the base of karrio 's abstraction. A Metadata declared at karrio/mappers/[carrier_name]/__init__ specifies the integration classes required to define a karrio compatible extension.

karrio/mappers/[carrier_name]/__init__.py
from karrio.core.metadata import Metadata

from karrio.mappers.[carrier_name].mapper import Mapper
from karrio.mappers.[carrier_name].proxy import Proxy
from karrio.mappers.[carrier_name].settings import Settings
import karrio.providers.[carrier_name].units as units


METADATA = Metadata(
id="[carrier_name]", # e.g: "ddp_uk"
label="[Carrier Name]", # e.g: "DDP UK"

# Integrations
Mapper=Mapper,
Proxy=Proxy,
Settings=Settings,

# Data Units (Optional...)
options=units.ShippingOption, # Enum of Shipping options supported by the carrier
package_presets=units.PackagePresets, # Enum of parcel presets/templates
services=units.ShippingService, # Enum of Shipping services supported by the carrier

is_hub=False # True if the carrier is a hub like (EasyPost, Shippo, Postmen...)
)
Module convention

Three modules are required to create a karrio extension.

  • karrio.mappers.[carrier_name]
  • karrio.providers.[carrier_name]
  • karrio.schemas.[carrier_name]
karrio.mappers.[carrier_name]

This is where the karrio abstract classes are implemented. Also, the Metadata require to identified the extension is also provided there.

on runtime, karrio retrieves all mappers by going trought the karrio.mappers modules

karrio.providers.[carrier_name]

This is where the mapping between karrio Unified API data is mapped on the carrier data type corresponding requests

karrio.schemas.[carrier_name]

This is where the your carrier generated python data types are stored.

File structure

The carrier extension package folder structure looks like this

modules/connectors/
[carrier_name]/
├── setup.py
├── generate
├── schemas
| └── error_response.json
| ├── rate_response.json
| ├── rate_request.json
| └── ...
└── karrio
│ ├── mappers
│ │ └── [carrier_name]
│ │ ├── __init__.py
│ │ ├── mapper.py
│ │ ├── proxy.py
│ │ └── settings.py
│ ├── providers
│ │ └── [carrier_name]
│ │ ├── __init__.py
│ │ ├── address.py
│ │ ├── error.py
│ │ ├── pickup
│ │ │ ├── __init__.py
│ │ │ ├── cancel.py
│ │ │ ├── create.py
│ │ │ └── update.py
│ │ ├── rate.py
│ │ ├── shipment
│ │ │ ├── __init__.py
│ │ │ ├── cancel.py
│ │ │ └── create.py
│ │ ├── tracking.py
│ | ├── units.py
│ | └── utils.py
│ └── schemas
│ └── [carrier_name]
│ ├── __init__.py
│ ├── error_response.py
│ ├── rate_request.py
│ ├── rate_response.py
│ └── ....
└── tests
├── __init__.py
└── [carrier_name]
├── __init__.py
├── fixture.py
├── test_rate.py
└── ...
info

Note that pickup and shipment modules are directories since there are often many sub to integrate such as create, cancel...


2. Generate Python data types

Karrio uses some cool open source projects to generate API schema datatypes.

info

We strongly believe in types because with a good IDE setup, they help you understand the datatypes available and makes maintenance easy.

Preapre the API schema files
  • For JSON APIs

First, you need to create .json files based on carrier API documentation under the [carrier_name]/schemas directory. Check the amazon_mws schemas files as examples.

Then, update the generate script to include the schema files list. Check the amazon_mws schema scripting as an example.

  • For XML/SOAP APIs

First, you need to collect .xsd files from the carrier API documentation and add them to the [carrier_name]/schemas directory. Check the canadapost schemas files as examples.

Then, update the generate script to include the schema files list. Please check the canadapost schema package as an example.

tip

If the carrier is SOAP based with .wsdl files, you need to extract the <xs:schema ...> sections into .xsd files.

Generate Python datatypes

To generate the Python data types, you need to run the generate script.

For a JSON API

Karrio uses a fork of quicktype to generate Python dataclasses from the jstruct library to turn JSON request and response samples.

warning

To use quicktype, you need to build a the custom docker image we have prepared for with our fork.

Terminal
./bin/build-tool-image
For a XML/SOAP API

Karrio uses the generateDS project cli to turn XML files and XML schemas from SOAP webservices into Python classes.

tip

generateDS is installed along with karrio' dev dependencies. So it should already be available.

Terminal
./bin/run-generate-on modules/connectors/[carrier_name]

3. Define carrier connection settings

The settings class is where the carrier connection settings are defined.

karrio/mappers/[carrier_name]/settings.py
"""Karrio Canada post client settings."""

import attr
from karrio.providers.canadapost import Settings as BaseSettings


@attr.s(auto_attribs=True)
class Settings(BaseSettings):
"""Canada post connection settings."""

username: str
password: str
customer_number: str = None
contract_id: str = None
language: str = "en"

id: str = None
test_mode: bool = False
carrier_id: str = "canadapost"
account_country_code: str = "CA"
metadata: dict = {}
config: dict = {}


4. Set up API calls under proxy

The proxy class is the carrier API client that makes the HTTP requests to the carrier API endpoints.

[carrier_name]/karrio/mappers/[carrier_name]/proxy.py
"""Karrio Australia Post client proxy."""

import karrio.lib as lib
import karrio.api.proxy as proxy
import karrio.mappers.australiapost.settings as provider_settings


class Proxy(proxy.Proxy):
settings: provider_settings.Settings

def get_rates(self, request: lib.Serializable) -> lib.Deserializable[str]:
response = lib.request(
url=f"{self.settings.server_url}/shipping/v1/prices/items",
data=lib.to_json(request.serialize()),
trace=self.trace_as("json"),
method="POST",
headers={
"Accept": "application/json",
"Content-Type": "application/json",
"Account-Number": self.settings.account_number,
"Authorization": f"Basic {self.settings.authorization}",
},
)

return lib.Deserializable(response, lib.to_dict)

# ....

def get_tracking(self, request: lib.Serializable) -> lib.Deserializable[str]:
query = request.serialize()
tracking_ids = ",".join(query["tracking_ids"])
response = lib.request(
url=f"{self.settings.server_url}/shipping/v1/track?tracking_ids={tracking_ids}",
trace=self.trace_as("json"),
method="GET",
headers={
"Accept": "application/json",
"Content-Type": "application/json",
"Account-Number": self.settings.account_number,
"Authorization": f"Basic {self.settings.authorization}",
},
)

return lib.Deserializable(response, lib.to_dict)


5. Implement the Mapper functions

The mapper function implementations consists of instantiating carrier specific request data types assigning.

info

The mapping function instantiates the carrier data types like a tree to offer a global view and simplify the mental relation between the code and the formatted data output based on the schema.

import karrio.schemas.amazon_shipping.rate_request as amazon
from karrio.schemas.amazon_shipping.rate_response import ServiceRate

import typing
import karrio.lib as lib
import karrio.core.units as units
import karrio.core.models as models
import karrio.core.errors as errors
import karrio.providers.amazon_shipping.error as provider_error
import karrio.providers.amazon_shipping.units as provider_units
import karrio.providers.amazon_shipping.utils as provider_utils


def parse_rate_response(
_response: lib.Deserializable[dict],
settings: provider_utils.Settings,
) -> typing.Tuple[typing.List[models.RateDetails], typing.List[models.Message]]:
response = _response.deserialize()
errors: typing.List[models.Message] = sum(
[
provider_error.parse_error_response(data, settings)
for data in response.get("errors", [])
],
[],
)
rates = [
_extract_details(data, settings) for data in response.get("serviceRates", [])
]

return rates, errors


def _extract_details(
data: dict,
settings: provider_utils.Settings,
) -> models.RateDetails:
rate = lib.to_object(ServiceRate, data)
transit = (
lib.to_date(rate.promise.deliveryWindow.start, "%Y-%m-%dT%H:%M:%S.%fZ").date()
- lib.to_date(rate.promise.receiveWindow.end, "%Y-%m-%dT%H:%M:%S.%fZ").date()
).days

return models.RateDetails(
carrier_id=settings.carrier_id,
carrier_name=settings.carrier_name,
service=provider_units.Service.map(rate.serviceType).name_or_key,
total_charge=lib.to_decimal(rate.totalCharge.value),
currency=rate.totalCharge.unit,
transit_days=transit,
meta=dict(
service_name=rate.serviceType,
),
)


def rate_request(payload: models.RateRequest, settings: provider_utils.Settings) -> lib.Serializable:
shipper = lib.to_address(payload.shipper)
recipient = lib.to_address(payload.recipient)
packages = lib.to_packages(payload.parcels)
options = lib.to_shipping_options(payload.options)
services = lib.to_services(payload.services, provider_units.Service)

request = amazon.RateRequest(
shipFrom=amazon.Ship(
name=shipper.person_name,
city=shipper.city,
addressLine1=shipper.street,
addressLine2=shipper.address_line2,
stateOrRegion=shipper.state_code,
email=shipper.email,
copyEmails=lib.join(shipper.email),
phoneNumber=shipper.phone_number,
),
shipTo=amazon.Ship(
name=recipient.person_name,
city=recipient.city,
addressLine1=recipient.street,
addressLine2=recipient.address_line2,
stateOrRegion=recipient.state_code,
email=recipient.email,
copyEmails=lib.join(recipient.email),
phoneNumber=recipient.phone_number,
),
serviceTypes=list(services),
shipDate=lib.fdatetime(
options.shipment_date.state, "%Y-%m-%d", "%Y-%m-%dT%H:%M:%S.%fZ"
),
containerSpecifications=[
amazon.ContainerSpecification(
dimensions=amazon.Dimensions(
height=package.height.IN,
length=package.length.IN,
width=package.width.IN,
unit="IN",
),
weight=amazon.Weight(
value=package.weight.LB,
unit="LB",
),
)
for package in packages
],
)

return lib.Serializable(request, lib.to_dict)


6. Add a database model for karrio server

The database model is a Django model that represents the carrier settings in the database.

modules/core/karrio/server/providers/extensions/[carrier_name].py
import django.db.models as models
import karrio.server.providers.models as providers


class CanadaPostSettings(providers.Carrier):
class Meta:
db_table = "canada-post-settings"
verbose_name = "Canada Post Settings"
verbose_name_plural = "Canada Post Settings"

username = models.CharField(max_length=200)
password = models.CharField(max_length=200)
customer_number = models.CharField(max_length=200, blank=True, null=True)
contract_id = models.CharField(max_length=200, blank=True, null=True)

@property
def carrier_name(self) -> str:
return "canadapost"


SETTINGS = CanadaPostSettings

Once the model is defined, you need to run the migrations to create the database table.

Terminal
# create the migration file
karrio makemigrations

# apply the migration
karrio migrate

7. Add the carrier configuration to the dashboard

This is the final step where you make your custom carrier configurable in the karrio dashboard.

Now that we have a database model, we can regenerate karrio's API schemas and Typescript typings.

  • Karrio Shipping REST API schema and Typescript typings
Terminal
# generate the REST OpenAPI schemas
./bin/server gen:openapi
Terminal
# generate the Typescript typings for the REST API
./bin/generate-openapi-types
  • Karrio management GraphQL API schema and Typescript typings
Terminal
# generate the GraphQL schema
./bin/server gen:graph

When a GraphQL schema is generated, you need to update the carrier connections queries for typings to be generated for them.

Update the user connections query fragment
packages/types/graphql/queries.ts
export const GET_USER_CONNECTIONS = gql`query get_user_connections {
user_connections {
__typename
... on AlliedExpressSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
config
username
password
account
service_type
}
... on AlliedExpressLocalSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
config
username
password
account
service_type
}
... on AmazonShippingSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
seller_id
developer_id
mws_auth_token
aws_region
config
}
... on AramexSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
username
password
account_pin
account_entity
account_number
account_country_code
config
}
... on AsendiaUSSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
username
password
account_number
api_key
config
}
... on AustraliaPostSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
api_key
password
account_number
config
}
... on BoxKnightSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
username
password
config
metadata
}
... on BelgianPostSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
config
account_id
passphrase
services {
id
active
service_name
service_code
carrier_service_code
description
currency
transit_days
transit_time
max_weight
max_width
max_height
max_length
weight_unit
dimension_unit
domicile
international
zones {
label
rate
min_weight
max_weight
transit_days
transit_time
radius
latitude
longitude
cities
postal_codes
country_codes
}
}
rate_sheet {
id
name
slug
carrier_name
metadata
}
}
... on CanadaPostSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
username
password
customer_number
contract_id
config
}
... on CanparSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
username
password
config
}
... on ChronopostSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
password
account_number
account_country_code
config
}
... on ColissimoSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
config
password
contract_number
laposte_api_key
services {
id
active
service_name
service_code
carrier_service_code
description
currency
transit_days
transit_time
max_weight
max_width
max_height
max_length
weight_unit
dimension_unit
domicile
international
zones {
label
rate
min_weight
max_weight
transit_days
transit_time
radius
latitude
longitude
cities
postal_codes
country_codes
}
}
rate_sheet {
id
name
slug
carrier_name
metadata
}
}
... on DHLParcelDESettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
username
password
customer_number
dhl_api_key
tracking_consumer_key
tracking_consumer_secret
config
services {
id
active
service_name
service_code
carrier_service_code
description
currency
transit_days
transit_time
max_weight
max_width
max_height
max_length
weight_unit
dimension_unit
domicile
international
zones {
label
rate
min_weight
max_weight
transit_days
transit_time
radius
latitude
longitude
cities
postal_codes
country_codes
}
}
rate_sheet {
id
name
slug
carrier_name
metadata
}
}
... on DHLExpressSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
site_id
password
account_number
account_country_code
config
}
... on DHLPolandSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
config
username
password
account_number
services {
id
active
service_name
service_code
carrier_service_code
description
currency
transit_days
transit_time
max_weight
max_width
max_height
max_length
weight_unit
dimension_unit
domicile
international
zones {
label
rate
min_weight
max_weight
transit_days
transit_time
radius
latitude
longitude
cities
postal_codes
country_codes
}
}
rate_sheet {
id
name
slug
carrier_name
metadata
}
}
... on DHLUniversalSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
consumer_key
consumer_secret
config
}
... on DicomSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
username
password
billing_account
config
}
... on DPDSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
config
capabilities
delis_id
password
depot
account_country_code
services {
active
currency
description
dimension_unit
domicile
id
international
max_height
max_length
max_weight
max_width
service_code
service_name
carrier_service_code
transit_days
transit_time
weight_unit
zones {
cities
postal_codes
country_codes
label
latitude
longitude
max_weight
min_weight
radius
rate
transit_days
transit_time
}
}
rate_sheet {
id
name
slug
carrier_name
metadata
}
}
... on DPDHLSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
username
password
app_id
app_token
zt_id
zt_password
account_number
config
services {
id
active
service_name
service_code
carrier_service_code
description
currency
transit_days
transit_time
max_weight
max_width
max_height
max_length
weight_unit
dimension_unit
domicile
international
zones {
label
rate
min_weight
max_weight
transit_days
transit_time
radius
latitude
longitude
cities
postal_codes
country_codes
}
}
rate_sheet {
id
name
slug
carrier_name
metadata
}
}
... on EShipperSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
username
password
config
}
... on EasyPostSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
api_key
config
}
... on FedexSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
account_number
api_key
secret_key
track_api_key
track_secret_key
account_country_code
config
}
... on FedexWSSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
account_number
password
meter_number
user_key
account_country_code
config
}
... on FreightcomSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
username
password
config
}
... on GenericSettingsType {
id
carrier_id
carrier_name
display_name
custom_carrier_name
account_number
test_mode
active
metadata
config
capabilities
account_country_code
services {
id
active
service_name
service_code
carrier_service_code
description
currency
transit_days
transit_time
max_weight
max_width
max_height
max_length
weight_unit
dimension_unit
domicile
international
zones {
label
rate
min_weight
max_weight
transit_days
transit_time
radius
latitude
longitude
cities
postal_codes
country_codes
}
}
label_template {
id
slug
template
template_type
shipment_sample
width
height
}
rate_sheet {
id
name
slug
carrier_name
metadata
}
}
... on GEODISSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
api_key
identifier
language
config
}
... on LaPosteSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
api_key
lang
config
}
... on Locate2uSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
config
account_country_code
client_id
client_secret
}
... on NationexSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
api_key
customer_id
billing_account
language
config
}
... on PurolatorSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
username
password
account_number
user_token
config
}
... on RoadieSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
api_key
config
}
... on RoyalMailSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
client_id
client_secret
config
}
... on SendleSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
sendle_id
api_key
account_country_code
config
}
... on TGESettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
config
capabilities
username
password
api_key
toll_username
toll_password
my_toll_token
my_toll_identity
account_code
}
... on TNTSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
username
password
account_number
account_country_code
config
}
... on UPSSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
client_id
client_secret
account_number
account_country_code
config
}
... on USPSSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
username
password
mailer_id
customer_registration_id
logistics_manager_mailer_id
config
}
... on USPSInternationalSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
metadata
capabilities
username
password
mailer_id
customer_registration_id
logistics_manager_mailer_id
config
}
... on Zoom2uSettingsType {
id
carrier_id
carrier_name
display_name
test_mode
active
capabilities
metadata
config
account_country_code
api_key
}
}
}
`;

Only then you can generate the Typescript typings for the GraphQL API.

Terminal
./bin/generate-graphql-types

Once the typings are updated to reflect the your new integration settings, you can now add the carrier configuration to the dashboard.

packages/ui/modals/connect-provider-modal.tsx

function fieldState(carrier_name: CarrierNameType, property: string) {
const field = (
({
[CarrierSettingsCarrierNameEnum.AlliedExpress]: [["carrier_id", true], ["username", true], ["password", true], ["account", false], ["service_type", false]],
[CarrierSettingsCarrierNameEnum.AmazonShipping]: [["carrier_id", true], ["seller_id", true], ["developer_id", true], ["mws_auth_token", true], ["aws_region"]],
// ...
[CarrierSettingsCarrierNameEnum.[CarrierName]]: [["carrier_id", true], ["username", true], ["password", true], /* ... */],
// ...
[CarrierSettingsCarrierNameEnum.Zoom2u]: [["carrier_id", true], ["api_key", true], ["account_country_code"]],
[NoneEnum.none]: [],
}[carrier_name] || [])
.find(([_, ...__]) => _ === property) || []
);
// ...
}