Skip to main content

Create Endpoint Feature - Test Scenarios Documentation

Overview​

This document provides comprehensive test scenarios for the Create Endpoint feature with JSONata API Connector. Each scenario includes step-by-step configuration, expected behavior, and validation points.

Table of Contents​

  1. Scenario 1: Successful Endpoint Creation (OpenAI-style API)
  2. Scenario 2: Wrong API Key (401/403 Error)
  3. Scenario 3: Incorrect Response Template (200 but Template Fails)
  4. Scenario 4: Incorrect Request Template (400 Error)
  5. Scenario 5: Domain Not Whitelisted
  6. Scenario 5: Domain Not Whitelisted

Scenario 1: Successful Endpoint Creation (OpenAI-style API)​

Description​

Complete successful flow where all configurations are correct, API key is valid, and responses are properly extracted.

Step 1: Basic Information​

Endpoint Name: 'OpenAI GPT-4 Endpoint'
Description: 'GPT-4 model endpoint for testing'

Step 2: Connection Details​

API URL: 'https://api.openai.com/v1/chat/completions'
Connection Type: 'jsonata-api-connector'
Timeout: 30 seconds
Max Attempts: 3
Max Calls Per Second: 10
Max Concurrency: 5

Step 3: Authentication​

Auth Type: "Bearer Token"
Token Value Type: "Fixed"
API Token: "sk-proj-valid-api-key-here"
Additional Headers: (optional)
  - Content-Type: application/json

Step 4: Request/Response Configuration​

Request Template (JSONata):

{
  "model": "gpt-4",
  "messages": [
    {
      "role": "user",
      "content": prompt
    }
  ],
  "temperature": 0.7
}

Response Template (JSONata):

choices[0].message.content

Expected Test Connection Results​

Prompt 1: "Hi, what do you do?"

{
  "statusCode": 200,
  "response": "I'm an AI assistant created by OpenAI. I can help you with...",
  "full_response": {
    "choices": [
      {
        "message": {
          "content": "I'm an AI assistant created by OpenAI. I can help you with..."
        }
      }
    ]
  },
  "duration": 1234
}

UI Display:

  • ✅ Green border
  • HTTP 200 badge (green)
  • Extracted Response: Shows the extracted text
  • Full API Response: Shows complete JSON response
  • Save button: Enabled, saves as ACTIVE

Scenario 2: Wrong API Key (401/403 Error)​

Description​

User provides an invalid or expired API key. The endpoint should properly display the authentication error from the API.

Configuration​

Same as Scenario 1, but with:

API Token: 'sk-proj-invalid-key-12345'

Expected Test Connection Results​

Prompt 1: "Hi, what do you do?"

{
  "statusCode": 403,
  "response": "{\"error\":{\"message\":\"Incorrect API key provided\",\"type\":\"invalid_request_error\"}}",
  "full_response": {
    "error": {
      "message": "Incorrect API key provided",
      "type": "invalid_request_error"
    }
  },
  "error": "HTTP error",
  "duration": 234
}

UI Display:

  • ❌ Red border
  • HTTP 403 badge (red)
  • Error: "HTTP error"
  • Error Response: Shows full error JSON
{
  "error": {
    "message": "Incorrect API key provided",
    "type": "invalid_request_error"
  }
}

Save Behavior:

  • Shows dialog: "Connection test failed. Save as INCOMPLETE?"
  • If user confirms: Saves with status INCOMPLETE
  • If user cancels: Returns to form

Scenario 3: Incorrect Response Template (200 but Template Fails)​

Description​

API key is valid and API returns 200, but the JSONata response template cannot extract data due to incorrect path.

Configuration​

Same as Scenario 1, but with:

Response Template (JSONata) - WRONG PATH:

data.answer

(Wrong because OpenAI uses choices[0].message.content, not data.answer)

Expected Test Connection Results​

Prompt 1: "Hi, what do you do?"

{
  "statusCode": 200,
  "response": "Failed to extract response using template: JSONata evaluation failed for response_template: path 'data' not found",
  "full_response": {
    "choices": [
      {
        "message": {
          "content": "I'm an AI assistant..."
        }
      }
    ]
  },
  "error": "Template extraction failed",
  "duration": 1234
}

UI Display:

  • ⚠️ Red border (marked as error)
  • HTTP 200 badge (red background) - API succeeded but template failed
  • Error Message: "Failed to extract response using template: JSONata evaluation failed..."
  • Error: "Template extraction failed"
  • Error Response: Shows the actual successful API response
{
  "choices": [
    {
      "message": {
        "content": "I'm an AI assistant..."
      }
    }
  ]
}

Key Learning: User can see:

  • ✅ API call succeeded (200)
  • ❌ Template is wrong
  • 👀 Actual response structure to fix the template

Save Behavior: Same as Scenario 2 - shows INCOMPLETE dialog

Scenario 4: Incorrect Request Template (400 Error)​

Description​

Request template generates invalid JSON or missing required fields, causing API to return 400 Bad Request.

Configuration​

Same as Scenario 1, but with:

Request Template (JSONata) - MISSING REQUIRED FIELD:

{
  "messages": [
    {
      "role": "user",
      "content": prompt
    }
  ]
}

(Missing required model field)

Expected Test Connection Results​

Prompt 1: "Hi, what do you do?"

{
  "statusCode": 400,
  "response": "{\"error\":{\"message\":\"you must provide a model parameter\",\"type\":\"invalid_request_error\"}}",
  "full_response": {
    "error": {
      "message": "you must provide a model parameter",
      "type": "invalid_request_error"
    }
  },
  "error": "HTTP error",
  "duration": 123
}

UI Display:

  • ❌ Red border
  • HTTP 400 badge (red)
  • Error: "HTTP error"
  • Error Response: Shows validation error from API

Scenario 5: Domain Whitelist with GitLab Notification​

Description​

User creates an endpoint with a domain that is not in the organization's whitelist. The system allows the save to proceed but automatically triggers a GitLab webhook to notify the DevOps team about the non-whitelisted domain. This is a notification system, not a blocking mechanism.

How It Works​

  1. User fills out endpoint configuration with a non-whitelisted domain
  2. User clicks "Test Connection" - test executes normally
  3. User clicks "Save Endpoint"
  4. Backend validates domain against whitelist (from AWS Parameter Store)
  5. If domain is NOT whitelisted:
    • ✅ Endpoint is saved successfully (status: ACTIVE or INCOMPLETE based on test results)
    • 📧 GitLab webhook is triggered to notify DevOps team
    • 📝 Warning is logged in backend logs
  6. If domain IS whitelisted:
    • ✅ Endpoint is saved successfully
    • ℹ️ Debug log confirms domain validation passed

Key Points​

  • Non-blocking: Domain validation does NOT prevent saving the endpoint
  • Notification-based: Uses GitLab CI/CD pipeline trigger to alert DevOps
  • Automatic: No manual intervention needed from user
  • Transparent: User sees normal save success, DevOps gets notified in background

Sub-Scenario 5A: Save Endpoint with Non-Whitelisted Domain​

Configuration​

API URL: 'https://unauthorized-api.example.com/v1/chat'
# All other settings same as Scenario 1
```yaml
API URL: 'https://new-partner-api.example.com/v1/chat'
API Key: 'valid-api-key-12345'
# All other settings same as Scenario 1

User Flow​

  1. User fills out all endpoint configuration steps
  2. User tests connection - Test succeeds (assuming correct API key and templates)
  3. User clicks "Save Endpoint"
  4. Backend saves endpoint to database successfully
  5. Backend validates domain: new-partner-api.example.com → NOT in whitelist
  6. Backend triggers GitLab webhook with domain information
  7. Endpoint appears in list with status ACTIVE

GitLab Pipeline Triggered:

Pipeline Variables:
  TRIGGER_TYPE: 'domain_whitelist_request'
  ENVIRONMENT: 'dev' # or "uat", "prod"
  DOMAIN: 'new-partner-api.example.com'

DevOps Team Notification:

  • GitLab CI/CD pipeline runs automatically
  • Creates merge request to add domain to whitelist
  • Notifies DevOps team via Slack/email
  • Team reviews domain and approves/rejects

User Experience​

Frontend Display:

  • ✅ Green success toast: "Endpoint created successfully"
  • ✅ Endpoint appears in list with ACTIVE status
  • ✅ No error messages shown to user
  • ✅ User can immediately start using the endpoint

Key Point: User is NOT blocked from saving the endpoint. The whitelist is for auditing and approval workflow, not for blocking saves.

Sub-Scenario 5B: Save Endpoint with Whitelisted Domain​

Configuration​

"api.anthropic.com", "your-internal-api.company.com" ]

Configuration​

API URL: 'https://api.openai.com/v1/chat/completions'
API Key: 'sk-proj-valid-key-12345'
# Domain is already in whitelist

User Flow​

  1. User fills out all endpoint configuration
  2. User tests connection - Test succeeds
  3. User clicks "Save Endpoint"
  4. Backend saves endpoint to database
  5. Backend validates domain: api.openai.com → IN whitelist ✅
  6. No GitLab webhook triggered (domain already approved)
  7. User sees success message: "Endpoint created successfully"

Backend Logs​

[DEBUG] Domain validation passed for URI: https://api.openai.com/v1/chat/completions
  domain: "api.openai.com"

[LOG] Saved to database with id abc-123-def

Key Point: No additional notifications sent when domain is already whitelisted.

Testing Checklist for Scenario 5​

  • Save endpoint with non-whitelisted domain → Should succeed
  • Verify GitLab webhook is triggered
  • Check backend logs for warning message
  • Verify endpoint appears in list with correct status
  • Save endpoint with whitelisted domain → No webhook triggered
  • Test wildcard pattern matching (*.gov.sg)
  • Test exact domain matching
  • Verify Parameter Store is queried on each validation
  • Test with empty whitelist (should allow all)
  • Verify webhook failure doesn't block save

Summary Matrix​

ScenarioStatus CodeTemplate WorksResultSave StatusGitLab Webhook
1. Success200✅Green, shows responseACTIVENo
2. Wrong API Key403N/ARed, shows API errorINCOMPLETENo
3. Wrong Template200❌Red, shows template error + API responseINCOMPLETENo
4. Bad Request400N/ARed, shows validation errorINCOMPLETENo
5A. Non-Whitelisted Domain (Save)N/AN/ASuccess, saves normallyACTIVE/INCOMPLETEYes ✅
5B. Whitelisted Domain (Save)N/AN/ASuccess, saves normallyACTIVE/INCOMPLETENo

Testing Checklist​

For Each Scenario:​

  • Navigate through all 4 steps without errors
  • Click "Test Connection" button
  • Verify both test prompts show correct status
  • Check status code badge color and value
  • Verify extracted response or error message displays correctly
  • Expand "Full API Response" and verify JSON structure
  • Check duration is displayed
  • Click "Save Endpoint"
  • Verify appropriate save behavior (ACTIVE or INCOMPLETE dialog)
  • Confirm endpoint is created with correct status in database

Edge Cases to Test:​

  • Empty required fields (should show validation errors)
  • Very long responses (should handle scrolling)
  • Special characters in endpoint name
  • Unicode/emoji in prompts and responses
  • Concurrent test connections (both prompts in parallel)
  • Browser refresh during test connection
  • Network disconnection during test