Feat/connector/shopify

[Jayyk09]

Comprehensive Shopify Source Connector Implementation

Overview

This PR implements a comprehensive Shopify source connector that enables users to sync e-commerce data from Shopify stores into Airweave. The connector utilizes Shopify's Admin GraphQL API (v2025-01) to ensure efficient data retrieval and supports multiple authentication methods, including direct access tokens and the full OAuth 2.0 browser flow.

Key Features

1. Flexible Authentication

The connector supports two primary methods for connecting to a Shopify store:

• Direct Authentication: Users can provide an access token generated via a Shopify Custom App.
• OAuth Browser Flow: Implements the full OAuth 2.0 authorization code flow using a shop-specific URL (https://{shop_name}.myshopify.com...).

2. Comprehensive Data Sync

We support syncing 8 core Shopify resource types. The hierarchy uses Shop as the parent breadcrumb for all child entities.

Resource	Entity Class	Description
Shop	ShopifyShopEntity	Store metadata (name, domain, currency, timezone).
Product	ShopifyProductEntity	Products including pricing, inventory, images, and variants.
Order	ShopifyOrderEntity	Orders with financial/fulfillment status and customer info.
Customer	ShopifyCustomerEntity	Customer profiles, addresses, and order history.
Collection	ShopifyCollectionEntity	Product collections (manual and automated).
DraftOrder	ShopifyDraftOrderEntity	Draft orders and invoices.
InventoryItem	ShopifyInventoryItemEntity	Inventory tracking and SKU details.
Location	ShopifyLocationEntity	Warehouses and fulfillment locations.
FulfillmentOrder	ShopifyFulfillmentOrderEntity	Fulfillment workflows and shipping details.

3. Robust GraphQL API Implementation

• API Version: Uses Shopify Admin GraphQL API 2025-01.
• Pagination: Implements cursor-based pagination to handle large datasets efficiently.
• Resilience: Intelligent retry logic with exponential backoff for handling rate limits (429) and timeouts.
• Configuration: Page size is configurable (Default: 50 items per request).

Architecture & Implementation

Source Configuration

The connector is defined with the following source decorator settings:

@source(
    name="Shopify",
    short_name="shopify",
    auth_methods=[DIRECT, OAUTH_BROWSER, AUTH_PROVIDER],
    oauth_type=OAuthType.ACCESS_ONLY, # Tokens don't expire until uninstall
    auth_config_class=ShopifyAuthConfig,          
    config_class="ShopifyConfig",
    labels=["E-commerce", "Sales"],
    rate_limit_level=RateLimitLevel.ORG,
)

OAuth Configuration

• Type: ACCESS_ONLY
• URL Template: https://{shop_name}.myshopify.com/admin/oauth/authorize
• Token URL: https://{shop_name}.myshopify.com/admin/oauth/access_token
• Scopes: read_products, read_orders, read_customers, read_inventory, read_locations, read_fulfillments, read_draft_orders

Design Highlights

• Validation Mode: Graceful handling of the OAuth callback even when shop_name is not yet available.
• Breadcrumb Hierarchy: The Shop entity serves as the parent breadcrumb for all child entities to maintain structure.
• Embeddable Fields: Specific entity fields are marked with embeddable=True to optimize for vector search.
• JSON Serialization: Complex nested data structures (e.g., addresses, rule sets, tags) are serialized as JSON strings to ensure flat structure compatibility.

Important Note: Deprecation Warning

Shopify Admin API Caveat: Legacy custom app development will be sunset on Jan 1, 2026. After this date, new integrations must use OAuth authentication. Previously created custom apps will continue to function, but this connector is built to support the future standard (OAuth) while maintaining backward compatibility for now.

Usage Guide

Method A: Direct Authentication

1. Go to Shopify Admin > Settings > Apps and sales channels > Develop apps.
2. Create a custom app and configure Admin API scopes (refer to the scopes list above).
3. Install the app and copy the Access Token.
4. In Airweave, enter the Shop Name and Access Token.

Method B: OAuth Authentication

1. Configure client_id and client_secret in your dev.integrations.yaml.
2. In Airweave, enter the Shop Name.
3. Complete the OAuth flow in the browser popup.
4. Sync begins automatically upon successful authorization.

Testing Considerations

• Prerequisites: Requires a Shopify store with either a valid access token or OAuth app credentials.
• Volume: Verify cursor-based pagination works correctly on stores with >50 items per resource.
• Rate Limiting: Test with rapid successive requests to verify exponential backoff logic.
• Data Integrity: Verify complex nested fields (addresses/variants) are serialized correctly.

Related Documentation

• Shopify Admin API Access Tokens
• Shopify GraphQL Admin API Reference

Testing:
The icon is broken because the frontend image is from Nov 10.

---

Summary by cubic

Adds a Shopify source connector to sync core e‑commerce data via the Admin GraphQL API (2025-01). Supports both direct access tokens and OAuth, with resilient pagination and rate‑limit handling.

• New Features

  • Shopify connector with OAuth (access_only) and direct token auth.
  • Syncs shop, products, orders, customers, collections, draft orders, inventory items, locations, and fulfillment orders.
  • Typed entities with shop breadcrumb; key fields marked embeddable for search.
  • Cursor-based pagination with retries on 429/timeouts; shop info bootstrap.
  • Config schemas for shop_name, api_version, resources; OAuth config added to dev.integrations.yaml.
  • Shopify app icon added to the dashboard.

• Migration

  • For OAuth: set client_id, client_secret, and required scopes in dev.integrations.yaml; enter the shop_name (myshopify subdomain).
  • For direct auth: provide the Admin API access token and shop_name.

^{Written for commit 9582dd6. Summary will update automatically on new commits.}

[cubic-dev-ai]

No issues found across 7 files