Skip to main content

Advanced Features

Insight Infusion - Finetune your own model​

The Insight Infusion feature allows users to adapt the base LLM to their specific use cases by training it with custom datasets.

Steps to Create a Fine-Tuning Job​

  1. Create a New Job
    1. Provide a Job Name.
    2. Specify the Input Column Name and Output Column Name.
    3. Enter the Dataset URL.
    4. If you do not have a dataset, you can generate one using the Dataset Generation feature described above.
  2. Configure a Run
    1. After creating a job, you can launch a Run with custom parameters such as:
      1. Rank
      2. Learning Rate
      3. Loss Function
      4. Other fine-tuning hyperparameters
  3. Monitor Run Results Once the run is completed, you can view:
    1. Metrics – Accuracy, loss values, and other metrics.
    2. GPU Usage – Resource utilization during training.
    3. Evaluation Results – Performance against validation/test sets.
    4. Hyperparameters – Parameters used for the run.
    5. Deployment Data – Information about model deployment readiness. Use the Deploy button to use this Custom Model Use the Revoke Deployment to end the deployment

Deployment Management​

From the Deployment Data section, you can:

  • Initiate a new deployment of the fine-tuned model.
  • Revoke an existing deployment if it is no longer required.

This workflow ensures that users can not only fine-tune their models but also seamlessly manage their deployment lifecycle within the platform. All trained adapters are automatically stored in the user's configured cloud storage. For example, if a Data Scientist has configured an S3 bucket, the adapters will be stored directly in that location.

Dataset Generation​

Before you can fine-tune a model, you need a well-structured dataset. TestingAide provides a Dataset Generator that allows you to automatically create high-quality datasets (with QA pairs) from your existing documentation.

Steps to Create a Dataset​

Step 1: Upload Documentation​

  • Upload your PDF or other supported files containing relevant information.

Step 2: Configure Dataset Options​

Select the contextual details that describe your application and testing environment. These options help generate datasets that are realistic, domain-specific, and aligned to your testing needs.

Project Context​
  • Application Type: Web, Mobile, API, Desktop, Embedded
  • Industry Domain: Finance, Healthcare, E-commerce, Education, Manufacturing
  • Team Size: Ranging from small (1–5 people) to large (50+ people)
  • Testing Framework: Selenium, Cypress, Playwright, Jest, Mocha
  • Testing Focus Areas: Functional, Performance, Security, Integration, Regression, Accessibility
Edge Cases & Complex Scenarios​
  • Error Handling Complexity: Simple → Critical
  • Data Boundary Scenarios: Min/Max values, Nulls, Invalid characters, Overflow
  • Network Resilience: High latency, disconnection, throttling, intermittent connectivity
  • State Management Edge Cases: Concurrent updates, session expiration, undo/redo, stale data
  • Critical Edge Case Categories: Race conditions, deadlocks, memory leaks, cascading failures
Legacy System Factors​
  • Application Age: New → Legacy (>15 years)
  • Technical Debt Level: Low → Critical
  • Documentation Quality: Excellent → Non-existent
  • Dependency Management: Well managed → Undocumented/critical issues
  • Legacy Challenges: Monolithic architectures, obsolete tech, hard-coded configs
Environment & Infrastructure Complexity​
  • Deployment Architecture: On-premise, Cloud-native, Hybrid, Serverless, Containerized
  • Environment Parity: High → Low (Dev vs Prod differences)
  • Data Complexity: Structured, Big Data, Streaming, Real-time Analytics
  • Third-party Integrations: None → Multiple complex & unreliable
  • Infrastructure Challenges: Auto-scaling, load balancing, caching, message queues
Compliance & Regulatory Requirements​
  • Regulations: GDPR, HIPAA, PCI DSS, CCPA, ISO 27001
  • Audit Needs: Annual, bi-annual, continuous
  • Data Sensitivity: Public → Highly Confidential/PII
  • Security Testing Depth: Basic scan → Red Team / Continuous monitoring
  • Compliance Focus: Access control, data privacy, encryption, logging, retention policies
Business Context & Constraints​
  • Testing Windows: Anytime, off-peak, weekends, scheduled downtime
  • Business Criticality: Low → Critical (business halting / revenue impact)
  • User Base Complexity: Small & simple → Large, global, technical/non-technical
  • Change Frequency: Daily → Quarterly
  • Business Risks: Financial impact, reputation risk, churn, competition, seasonal traffic, rapid scaling

Step 3: Generate Dataset​

  • Once you've configured all relevant options, click Generate.
  • TestingAide will create a structured dataset with QA pairs based on your inputs.

Step 4: View & Use Dataset​

  • The generated dataset can be viewed directly within the Generated Dataset section.
  • A dataset URL is also provided for easy copy-to-clipboard access, making it simple to plug into Insight Infusionjobs.

This setup ensures that your datasets are context-aware, domain-specific, and aligned with edge cases, giving you much better fine-tuning results compared to raw/manual data prep.

Concept Curation - Context Retrieval at Execution​

Users can create a Retrieval-Augmented Generation (RAG) pipeline using their existing documentation. The platform provides a drag-and-drop interface where each component is represented by its functional name, making the workflow intuitive to build.

RAG Components​

  1. Data Sources (S3, SharePoint,Google drive,Local Files)

    This is where your raw documents reside. Connectors fetch and ingest files from cloud storage (e.g., S3) or enterprise repositories (e.g., SharePoint).

  2. Text Splitter

    Breaks large documents into smaller chunks that fit within token limits.

    • Recursive Splitter: Splits intelligently at natural boundaries (paragraphs, sentences).
    • Character Splitter: Splits text at fixed character lengths.

  3. Embeddings

    Embeddings turn text into numerical representations that capture context and similarity. When you upload documents, each section is converted into embeddings.The options we provide are

    • OpenAI Embeddings – Fast and highly accurate, ideal if you want reliable semantic search with minimal setup.
    • Hugging Face Embeddings – Open-source models that can be run locally or on your own infrastructure, giving flexibility and control.

  4. Vector Database

    • Stores embeddings (numerical representations of text).
    • Enables fast semantic search to quickly retrieve relevant chunks.
    • pgvector: Postgres-based option.
    • Pinecone: Managed, scalable vector database.

  5. LLM (Large Language Model)

    • Acts as the reasoning engine.
    • Retrieves the most relevant document chunks from the vector database.
    • Generates natural-language answers to user queries based on the retrieved context.

Workflow​

  • Click on Add New

  • Using the drag-and-drop interface, you can assemble these components in sequence: Data Source → Text Splitter → Embeddings → Vector Database → LLM.

  • Each component functions according to its name, making setup straightforward even for first-time users.Once configured, the RAG pipeline allows users to query their documentation in natural language, with accurate, context-aware answers.
  • This RAG would also be used in test case genation
When to Use Concept Curation vs. Insight-Infusion?

Use Concept Curation if you have versatile or unstructured data (e.g., PDFs, documents, or mixed formats). RAG allows you to query this data on demand without retraining a model.

Use Insight-Infusion if you already have a well-structured, high-quality dataset and need the model to learn domain-specific patterns. Fine-tuning is more resource-intensive and time-consuming compared to RAG.