Pattern 6: Middleware

Use middleware when external systems can't authenticate directly to NetSuite or when you need data transformation.

---

How It Works

MIDDLEWARE PATTERN
─────────────────────────────────────────────────────────────────

External System                Middleware                 NetSuite
(No OAuth capability)          (AWS Lambda, etc)          (Secure)
       │                             │                        │
       │  Webhook (no auth)          │                        │
       │ ──────────────────────────▶ │                        │
       │                             │  REST API (OAuth 2.0)  │
       │                             │ ──────────────────────▶│
       │                             │                        │
       │                             │ ◀────────────────────── │
       │  Response                   │                        │
       │ ◀────────────────────────── │                        │
       │                             │                        │

Middleware handles:
├── Receive unauthenticated webhooks
├── Validate & transform data
├── Authenticate to NetSuite (OAuth 2.0)
├── Rate limiting & retries
└── Logging & monitoring

---

When to Use Middleware

Scenario	Reason
Legacy system can't do OAuth	Middleware handles authentication
Need data transformation	Convert formats before sending
Rate limiting required	Buffer requests to avoid limits
Multiple sources to one destination	Aggregate before sending
Need retries and dead-letter queue	Handle failures gracefully
Compliance logging required	Audit all data in transit
Multi-step orchestration	Chain multiple API calls

---

Middleware Options

Platform	Best For	Complexity
AWS Lambda	Simple transformations	Low
Azure Functions	Microsoft ecosystems	Low
Google Cloud Functions	GCP users	Low
Celigo	NetSuite integrations	Medium
Dell Boomi	Enterprise iPaaS	High
MuleSoft	Complex enterprise	High
Workato	Business users	Medium
Zapier	Simple automations	Low

---

Scenario A: AWS Lambda Middleware

Case: Shopify can't do NetSuite OAuth, use Lambda as middleware.

FLOW
─────────────────────────────────────────────────────────────────

Shopify (Order Created)
        │
        │ POST webhook (simple auth)
        ▼
┌─────────────────┐
│ AWS Lambda      │
│ ────────────    │
│ 1. Verify sig   │
│ 2. Transform    │
│ 3. Get OAuth    │
│ 4. Call RESTlet │
└────────┬────────┘
         │ POST (OAuth 2.0)
         ▼
┌─────────────────┐
│ NetSuite        │
│ RESTlet         │
│ (Create SO)     │
└─────────────────┘

AWS Lambda (Node.js):

// handler.js
const axios = require('axios');
const crypto = require('crypto');
const { getNetSuiteToken } = require('./auth');

exports.handler = async (event) => {
    console.log('Received webhook:', event.headers);

    // 1. Validate Shopify signature
    const signature = event.headers['x-shopify-hmac-sha256'];
    const body = event.body;

    if (!validateShopifySignature(body, signature)) {
        return { statusCode: 401, body: 'Invalid signature' };
    }

    // 2. Parse and transform payload
    const shopifyOrder = JSON.parse(body);
    const netsuitePayload = transformOrder(shopifyOrder);

    // 3. Get OAuth 2.0 token for NetSuite
    const token = await getNetSuiteToken();

    // 4. Send to NetSuite RESTlet
    try {
        const response = await axios.post(
            process.env.NETSUITE_RESTLET_URL,
            netsuitePayload,
            {
                headers: {
                    'Authorization': `Bearer ${token}`,
                    'Content-Type': 'application/json'
                },
                timeout: 30000
            }
        );

        console.log('NetSuite response:', response.data);

        return {
            statusCode: 200,
            body: JSON.stringify({
                success: true,
                netsuiteId: response.data.id
            })
        };

    } catch (error) {
        console.error('NetSuite error:', error.message);

        // Queue for retry using SQS
        await queueForRetry(netsuitePayload, error.message);

        return {
            statusCode: 202,
            body: JSON.stringify({
                success: false,
                queued: true,
                error: error.message
            })
        };
    }
};

function validateShopifySignature(body, signature) {
    const hmac = crypto
        .createHmac('sha256', process.env.SHOPIFY_SECRET)
        .update(body)
        .digest('base64');
    return hmac === signature;
}

function transformOrder(shopifyOrder) {
    return {
        externalId: shopifyOrder.id,
        orderNumber: shopifyOrder.order_number,
        customer: {
            email: shopifyOrder.customer.email,
            name: shopifyOrder.customer.first_name + ' ' + shopifyOrder.customer.last_name
        },
        items: shopifyOrder.line_items.map(item => ({
            sku: item.sku,
            quantity: item.quantity,
            price: item.price
        })),
        shipping: {
            address1: shopifyOrder.shipping_address.address1,
            city: shopifyOrder.shipping_address.city,
            zip: shopifyOrder.shipping_address.zip,
            country: shopifyOrder.shipping_address.country_code
        },
        total: shopifyOrder.total_price
    };
}

async function queueForRetry(payload, error) {
    const AWS = require('aws-sdk');
    const sqs = new AWS.SQS();

    await sqs.sendMessage({
        QueueUrl: process.env.RETRY_QUEUE_URL,
        MessageBody: JSON.stringify({
            payload: payload,
            error: error,
            timestamp: new Date().toISOString(),
            attempts: 1
        })
    }).promise();
}

OAuth 2.0 Token Management (auth.js):

// auth.js
const axios = require('axios');
const jwt = require('jsonwebtoken');
const fs = require('fs');

let cachedToken = null;
let tokenExpiry = null;

exports.getNetSuiteToken = async function() {
    // Return cached token if still valid
    if (cachedToken && tokenExpiry && new Date() < tokenExpiry) {
        return cachedToken;
    }

    // Build JWT assertion for client credentials flow
    const now = Math.floor(Date.now() / 1000);
    const privateKey = fs.readFileSync('./private-key.pem');

    const assertion = jwt.sign(
        {
            iss: process.env.NETSUITE_CLIENT_ID,
            scope: 'restlets',
            aud: process.env.NETSUITE_TOKEN_URL,
            iat: now,
            exp: now + 3600
        },
        privateKey,
        { algorithm: 'RS256' }
    );

    // Exchange assertion for access token
    const response = await axios.post(
        process.env.NETSUITE_TOKEN_URL,
        new URLSearchParams({
            grant_type: 'client_credentials',
            client_assertion_type: 'urn:ietf:params:oauth:client-assertion-type:jwt-bearer',
            client_assertion: assertion
        }),
        {
            headers: { 'Content-Type': 'application/x-www-form-urlencoded' }
        }
    );

    cachedToken = response.data.access_token;
    tokenExpiry = new Date(Date.now() + (response.data.expires_in - 60) * 1000);

    return cachedToken;
};

---

Scenario B: Multi-Step Orchestration

Case: Order from e-commerce requires: create customer, create order, reserve inventory.

FLOW
─────────────────────────────────────────────────────────────────

E-commerce Order
        │
        ▼
┌─────────────────┐
│ Step 1:         │
│ Find/Create     │ ──────▶ NetSuite Customer API
│ Customer        │
└────────┬────────┘
         │ customer ID
         ▼
┌─────────────────┐
│ Step 2:         │
│ Create Sales    │ ──────▶ NetSuite SO API
│ Order           │
└────────┬────────┘
         │ order ID
         ▼
┌─────────────────┐
│ Step 3:         │
│ Reserve         │ ──────▶ Warehouse API
│ Inventory       │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Step 4:         │
│ Send            │ ──────▶ Customer Email
│ Confirmation    │
└─────────────────┘

Lambda Orchestration:

exports.handler = async (event) => {
    const order = JSON.parse(event.body);
    const results = { steps: [] };

    try {
        // Step 1: Find or create customer
        const customerId = await findOrCreateCustomer(order.customer);
        results.steps.push({ step: 'customer', success: true, id: customerId });

        // Step 2: Create sales order
        const salesOrderId = await createSalesOrder(order, customerId);
        results.steps.push({ step: 'salesOrder', success: true, id: salesOrderId });

        // Step 3: Reserve inventory
        const reservationId = await reserveInventory(order.items, salesOrderId);
        results.steps.push({ step: 'inventory', success: true, id: reservationId });

        // Step 4: Send confirmation
        await sendConfirmation(order.customer.email, salesOrderId);
        results.steps.push({ step: 'confirmation', success: true });

        return {
            statusCode: 200,
            body: JSON.stringify({ success: true, results })
        };

    } catch (error) {
        results.error = error.message;

        // Compensating transaction - rollback if needed
        await rollback(results.steps);

        return {
            statusCode: 500,
            body: JSON.stringify({ success: false, results })
        };
    }
};

async function rollback(completedSteps) {
    // Reverse order of completed steps
    for (const step of completedSteps.reverse()) {
        try {
            switch (step.step) {
                case 'salesOrder':
                    await cancelSalesOrder(step.id);
                    break;
                case 'inventory':
                    await releaseReservation(step.id);
                    break;
                // Customers usually don't need rollback
            }
        } catch (e) {
            console.error('Rollback failed for', step.step, e.message);
        }
    }
}

---

Scenario C: Data Transformation Layer

Case: Transform complex XML from legacy system to NetSuite JSON.

// transform.js
const xml2js = require('xml2js');

exports.transformLegacyOrder = async function(xmlData) {
    // Parse XML
    const parser = new xml2js.Parser({ explicitArray: false });
    const legacyOrder = await parser.parseStringPromise(xmlData);

    // Map to NetSuite format
    const netsuiteOrder = {
        externalId: legacyOrder.ORDER.HEADER.ORDER_NO,

        // Customer mapping
        entity: await lookupCustomer(legacyOrder.ORDER.HEADER.CUSTOMER_CODE),

        // Date transformation (legacy: YYYYMMDD → ISO)
        trandate: transformDate(legacyOrder.ORDER.HEADER.ORDER_DATE),

        // Line items
        items: legacyOrder.ORDER.LINES.LINE.map(line => ({
            item: mapItemCode(line.PRODUCT_CODE),
            quantity: parseInt(line.QTY),
            rate: parseFloat(line.UNIT_PRICE),
            // Custom fields
            custcol_legacy_line_id: line.LINE_NO
        })),

        // Address transformation
        shipaddress: buildAddress(legacyOrder.ORDER.SHIPPING),

        // Custom body fields
        custbody_legacy_order_id: legacyOrder.ORDER.HEADER.ORDER_NO,
        custbody_source_system: 'LEGACY_ERP'
    };

    return netsuiteOrder;
};

function transformDate(legacyDate) {
    // YYYYMMDD → YYYY-MM-DD
    return legacyDate.slice(0, 4) + '-' +
           legacyDate.slice(4, 6) + '-' +
           legacyDate.slice(6, 8);
}

function buildAddress(shipping) {
    return [
        shipping.ATTN,
        shipping.ADDR1,
        shipping.ADDR2,
        shipping.CITY + ', ' + shipping.STATE + ' ' + shipping.ZIP,
        shipping.COUNTRY
    ].filter(Boolean).join('\n');
}

---

Retry Queue Architecture

RETRY ARCHITECTURE
─────────────────────────────────────────────────────────────────

Primary Flow:
Webhook ──▶ Lambda ──▶ NetSuite
                │
                │ On failure
                ▼
           ┌─────────┐
           │ SQS     │ ◀── Dead Letter Queue
           │ Queue   │     (after 3 retries)
           └────┬────┘
                │
                ▼
           ┌─────────┐
           │ Retry   │ ──▶ NetSuite
           │ Lambda  │
           └─────────┘

Retry Lambda:

exports.handler = async (event) => {
    for (const record of event.Records) {
        const message = JSON.parse(record.body);

        // Exponential backoff
        const delay = Math.pow(2, message.attempts) * 1000;

        try {
            // Wait before retry
            await sleep(delay);

            // Retry the API call
            await callNetSuite(message.payload);

            // Success - message will be deleted

        } catch (error) {
            message.attempts++;

            if (message.attempts >= 3) {
                // Move to dead letter queue
                throw new Error('Max retries exceeded');
            }

            // Re-queue with updated attempt count
            await requeue(message);
        }
    }
};

function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

---

Monitoring & Alerting

// Add CloudWatch metrics
const AWS = require('aws-sdk');
const cloudwatch = new AWS.CloudWatch();

async function recordMetric(name, value, unit = 'Count') {
    await cloudwatch.putMetricData({
        Namespace: 'NetSuiteIntegration',
        MetricData: [{
            MetricName: name,
            Value: value,
            Unit: unit,
            Dimensions: [{
                Name: 'Environment',
                Value: process.env.STAGE
            }]
        }]
    }).promise();
}

// Usage
await recordMetric('WebhooksReceived', 1);
await recordMetric('NetSuiteCallsSuccess', 1);
await recordMetric('NetSuiteCallsFailed', 1);
await recordMetric('ProcessingTime', 250, 'Milliseconds');

---

Security Best Practices

Practice	Implementation
Store secrets in Secrets Manager	AWS Secrets Manager, Azure Key Vault
Validate webhook signatures	HMAC verification
Use VPC for Lambda	Private subnets, NAT gateway
Enable CloudTrail	Audit API calls
Encrypt data at rest	KMS encryption
Rotate credentials	Automatic rotation
IP allowlisting	Restrict NetSuite access

---

Common Middleware Platforms

Celigo (NetSuite-Focused)

┌─────────────────────────────────────────────────────────────────┐
│ Celigo integrator.io                                            │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  Pros:                          Cons:                           │
│  • Pre-built NetSuite connectors • Subscription cost            │
│  • Visual flow builder          • Learning curve                │
│  • Error handling built-in      • Limited customization         │
│  • Monitoring dashboard         • Vendor lock-in                │
│                                                                 │
│  Best for: Medium complexity integrations                       │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Dell Boomi / MuleSoft (Enterprise)

┌─────────────────────────────────────────────────────────────────┐
│ Enterprise iPaaS                                                │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  Pros:                          Cons:                           │
│  • Enterprise-grade             • High cost                     │
│  • Many connectors              • Complex setup                 │
│  • Strong governance            • Requires specialists          │
│  • On-premise option            • Overkill for simple needs     │
│                                                                 │
│  Best for: Large enterprise with many systems                   │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

---

Real World Case: Banking FTP Integration

For comprehensive banking SFTP integration with VPS middleware, IP whitelisting, and ACK1/ACK2 patterns, see the dedicated guide:

Banking FTP Integration Guide - Complete implementation including:

• Full VPS middleware (when NetSuite IP not whitelisted)
• Hybrid approach (direct bank upload + VPS ACK bridge)
• VPS bash scripts and cron configuration
• NetSuite Scheduled Scripts for upload and ACK polling
• ACK1/ACK2 file patterns and status tracking

---

Decision Guide

WHEN TO USE MIDDLEWARE
─────────────────────────────────────────────────────────────────

Does the external system support OAuth?
    │
    ├── Yes ──▶ Direct integration (RESTlet/Suitelet)
    │
    └── No
            │
            ▼
        Need data transformation?
            │
            ├── Simple ──▶ SuiteScript handles it
            │
            └── Complex ──▶ Middleware recommended
                    │
                    ▼
                Volume considerations?
                    │
                    ├── Low (<100/day) ──▶ Lambda/Functions
                    │
                    └── High (1000s/day) ──▶ iPaaS platform

---

Related Patterns

• Receive - Direct webhook to NetSuite
• Push Out - NetSuite pushes to external
• File-Based - Alternative for legacy systems