# Quick Start: Knowledge Table

## 1. Introduction

This guide demonstrates how to use JamAI Base TypeScript SDK to upload and embed files into Knowledge Tables for AI-powered document processing and retrieval.

### What are Knowledge Tables?

Knowledge Tables are specialized tables in JamAI Base that provide hybrid-search capabilities through both full-text search (FTS) and vector embeddings:

* **Search Capabilities**:
  * **Full-Text Search (FTS)**: Traditional keyword-based search for exact and partial matches
  * **Semantic Search**: Vector embedding-based search for meaning and context
* **Document Processing**:
  * Automatically chunks documents into manageable segments
  * Generates vector embeddings for semantic understanding
  * Indexes content for full-text search
  * Preserves document structure (tables, layouts, etc) and metadata
* **Use Cases**:
  * Document retrieval using both keywords and semantic meaning
  * Question-answering agent
  * Content recommendation
  * Knowledge base search and discovery

### Supported File Types

The following file formats are supported:

* Text files: `.txt`, `.md`, `.csv`, `.tsv`
* Documents: `.doc`, `.docx`, `.pdf`
* Presentations: `.ppt`, `.pptx`
* Spreadsheets: `.xls`, `.xlsx`
* Markup/Data: `.xml`, `.html`, `.json`, `.jsonl`

### Prerequisites

Before starting, you'll need:

* Node.js 16.x or higher
* Project ID and Personal Access Token (PAT)
* Documents to process

## 2. Installation and Setup

### Installing Required Packages

```bash
npm install jamaibase dotenv
```

### Basic Configuration

```typescript
import JamAI from "jamaibase";
import * as dotenv from "dotenv";

// Load environment variables
dotenv.config();

const PROJECT_ID = "your_project_id";
const PAT = process.env.PAT || "your_PAT";

const client = new JamAI({
  projectId: PROJECT_ID,
  token: PAT,
});
```

## 3. Creating Your Knowledge Table

{% stepper %}
{% step %}

### Create a new table

Navigate to your JamAI Base knowledge tables tab and create a new knowledge table.
{% endstep %}

{% step %}

### Note table ID

After creating the table, note down the table ID for later use.
{% endstep %}
{% endstepper %}

## 4. Implementation

### 4.1 Complete Document Uploader Class

```typescript
import JamAI from "jamaibase";
import { File } from "formdata-node";
import * as fs from "fs";
import * as path from "path";

class DocumentUploader {
  private client: JamAI;

  constructor(projectId: string, pat: string) {
    // Initialize the document uploader
    this.client = new JamAI({
      projectId: projectId,
      token: pat,
    });
  }

  validateFile(filePath: string): boolean {
    // Validate if file exists and has supported format
    if (!fs.existsSync(filePath)) {
      throw new Error(`File not found: ${filePath}`);
    }

    const supportedTypes = [
      ".csv",
      ".tsv",
      ".txt",
      ".md",
      ".doc",
      ".docx",
      ".pdf",
      ".ppt",
      ".pptx",
      ".xls",
      ".xlsx",
      ".xml",
      ".html",
      ".json",
      ".jsonl",
    ];
    const fileExt = path.extname(filePath).toLowerCase();

    if (!supportedTypes.includes(fileExt)) {
      throw new Error(
        `Unsupported file format. Supported formats: ${supportedTypes.join(
          ", "
        )}`
      );
    }

    return true;
  }

  getMimeType(filePath: string): string {
    // Get MIME type of the file
    const mimeTypes: { [key: string]: string } = {
      ".csv": "text/csv",
      ".tsv": "text/tab-separated-values",
      ".txt": "text/plain",
      ".md": "text/markdown",
      ".doc": "application/msword",
      ".docx":
        "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
      ".pdf": "application/pdf",
      ".ppt": "application/vnd.ms-powerpoint",
      ".pptx":
        "application/vnd.openxmlformats-officedocument.presentationml.presentation",
      ".xls": "application/vnd.ms-excel",
      ".xlsx":
        "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
      ".xml": "application/xml",
      ".html": "text/html",
      ".json": "application/json",
      ".jsonl": "application/jsonl",
    };

    const fileExt = path.extname(filePath).toLowerCase();
    return mimeTypes[fileExt] || "application/octet-stream";
  }

  getOptimalChunkSettings(filePath: string): { size: number; overlap: number } {
    // Determine optimal chunk settings based on file type
    const fileExt = path.extname(filePath).toLowerCase();

    const settings: Record<string, { size: number; overlap: number }> = {
      // Text-based documents
      ".txt": { size: 1000, overlap: 200 },
      ".md": { size: 1000, overlap: 200 },
      ".csv": { size: 800, overlap: 150 },
      ".tsv": { size: 800, overlap: 150 },

      // Rich text documents
      ".doc": { size: 1200, overlap: 250 },
      ".docx": { size: 1200, overlap: 250 },
      ".pdf": { size: 1500, overlap: 300 },

      // Presentations
      ".ppt": { size: 1000, overlap: 200 },
      ".pptx": { size: 1000, overlap: 200 },

      // Spreadsheets
      ".xls": { size: 800, overlap: 150 },
      ".xlsx": { size: 800, overlap: 150 },

      // Markup/structured documents
      ".xml": { size: 1000, overlap: 200 },
      ".html": { size: 1000, overlap: 200 },
      ".json": { size: 800, overlap: 150 },
      ".jsonl": { size: 800, overlap: 150 },
    };

    return settings[fileExt] || { size: 1000, overlap: 200 };
  }

  async uploadDocument(
    filePath: string,
    tableId: string,
    customChunkSize?: number,
    customChunkOverlap?: number
  ): Promise<boolean> {
    try {
      // Validate file
      this.validateFile(filePath);

      // Get file information
      const fileName = path.basename(filePath);
      const fileStats = fs.statSync(filePath);
      const fileSize = fileStats.size;
      const mimeType = this.getMimeType(filePath);

      // Get chunk settings
      const settings = this.getOptimalChunkSettings(filePath);
      const chunkSize = customChunkSize || settings.size;
      const chunkOverlap = customChunkOverlap || settings.overlap;

      console.log(`Uploading: ${fileName}`);
      console.log(`File type: ${mimeType}`);
      console.log(`File size: ${(fileSize / 1024).toFixed(2)} KB`);
      console.log(`Chunk size: ${chunkSize}, Overlap: ${chunkOverlap}`);

      // Read file and create File object
      const fileBuffer = fs.readFileSync(filePath);
      const file = new File([fileBuffer], fileName, { type: mimeType });

      // Upload and embed file
      const response = await this.client.table.embedFile({
        file: file,
        table_id: tableId,
        chunk_size: chunkSize,
        chunk_overlap: chunkOverlap,
      });

      console.log(`Upload successful: ${fileName}`);
      return true;
    } catch (error) {
      console.error(`Error uploading ${filePath}: ${error}`);
      return false;
    }
  }
}
```

## 5. Complete Standalone Script

Save this as `knowledge_uploader.ts`:

```typescript
import JamAI from "jamaibase";
import { File } from "formdata-node";
import * as fs from "fs";
import * as path from "path";
import * as dotenv from "dotenv";

class DocumentUploader {
  private client: JamAI;

  constructor(projectId: string, pat: string) {
    this.client = new JamAI({
      projectId: projectId,
      token: pat,
    });
  }

  validateFile(filePath: string): boolean {
    if (!fs.existsSync(filePath)) {
      throw new Error(`File not found: ${filePath}`);
    }

    const supportedTypes = [
      ".csv",
      ".tsv",
      ".txt",
      ".md",
      ".doc",
      ".docx",
      ".pdf",
      ".ppt",
      ".pptx",
      ".xls",
      ".xlsx",
      ".xml",
      ".html",
      ".json",
      ".jsonl",
    ];
    const fileExt = path.extname(filePath).toLowerCase();

    if (!supportedTypes.includes(fileExt)) {
      throw new Error(
        `Unsupported file format. Supported formats: ${supportedTypes.join(
          ", "
        )}`
      );
    }

    return true;
  }

  getMimeType(filePath: string): string {
    const mimeTypes: { [key: string]: string } = {
      ".csv": "text/csv",
      ".tsv": "text/tab-separated-values",
      ".txt": "text/plain",
      ".md": "text/markdown",
      ".doc": "application/msword",
      ".docx":
        "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
      ".pdf": "application/pdf",
      ".ppt": "application/vnd.ms-powerpoint",
      ".pptx":
        "application/vnd.openxmlformats-officedocument.presentationml.presentation",
      ".xls": "application/vnd.ms-excel",
      ".xlsx":
        "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
      ".xml": "application/xml",
      ".html": "text/html",
      ".json": "application/json",
      ".jsonl": "application/jsonl",
    };

    const fileExt = path.extname(filePath).toLowerCase();
    return mimeTypes[fileExt] || "application/octet-stream";
  }

  getOptimalChunkSettings(filePath: string): { size: number; overlap: number } {
    const fileExt = path.extname(filePath).toLowerCase();

    const settings: { [key: string]: { size: number; overlap: number } } = {
      ".txt": { size: 1000, overlap: 200 },
      ".md": { size: 1000, overlap: 200 },
      ".csv": { size: 800, overlap: 150 },
      ".tsv": { size: 800, overlap: 150 },
      ".doc": { size: 1200, overlap: 250 },
      ".docx": { size: 1200, overlap: 250 },
      ".pdf": { size: 1500, overlap: 300 },
      ".ppt": { size: 1000, overlap: 200 },
      ".pptx": { size: 1000, overlap: 200 },
      ".xls": { size: 800, overlap: 150 },
      ".xlsx": { size: 800, overlap: 150 },
      ".xml": { size: 1000, overlap: 200 },
      ".html": { size: 1000, overlap: 200 },
      ".json": { size: 800, overlap: 150 },
      ".jsonl": { size: 800, overlap: 150 },
    };

    return settings[fileExt] || { size: 1000, overlap: 200 };
  }

  async uploadDocument(
    filePath: string,
    tableId: string,
    customChunkSize?: number,
    customChunkOverlap?: number
  ): Promise<boolean> {
    try {
      this.validateFile(filePath);

      const fileName = path.basename(filePath);
      const fileStats = fs.statSync(filePath);
      const fileSize = fileStats.size;
      const mimeType = this.getMimeType(filePath);

      const settings = this.getOptimalChunkSettings(filePath);
      const chunkSize = customChunkSize || settings.size;
      const chunkOverlap = customChunkOverlap || settings.overlap;

      console.log(`Uploading: ${fileName}`);
      console.log(`File type: ${mimeType}`);
      console.log(`File size: ${(fileSize / 1024).toFixed(2)} KB`);
      console.log(`Chunk size: ${chunkSize}, Overlap: ${chunkOverlap}`);

      const fileBuffer = fs.readFileSync(filePath);
      const file = new File([fileBuffer], fileName, { type: mimeType });

      const response = await this.client.table.embedFile({
        file: file,
        table_id: tableId,
        chunk_size: chunkSize,
        chunk_overlap: chunkOverlap,
      });

      console.log(`Upload successful: ${fileName}`);
      return true;
    } catch (error) {
      console.error(`Error uploading ${filePath}: ${error}`);
      return false;
    }
  }
}

async function processFolder(
  folderPath: string,
  uploader: DocumentUploader,
  tableId: string,
  chunkSize?: number,
  chunkOverlap?: number
): Promise<{ successful: string[]; failed: string[] }> {
  const successful: string[] = [];
  const failed: string[] = [];

  const files = fs.readdirSync(folderPath);

  for (const filename of files) {
    const filePath = path.join(folderPath, filename);
    if (fs.statSync(filePath).isFile()) {
      const success = await uploader.uploadDocument(
        filePath,
        tableId,
        chunkSize,
        chunkOverlap
      );
      if (success) {
        successful.push(filename);
      } else {
        failed.push(filename);
      }
    }
  }

  return { successful, failed };
}

// Main execution
async function main() {
  // Load environment variables
  dotenv.config();

  // Configuration (use environment variables or command-line args)
  const PROJECT_ID = process.env.JAMAI_PROJECT_ID || "your_project_id";
  const PAT = process.env.JAMAI_API_KEY || "your_PAT";
  const TABLE_ID = process.env.TABLE_ID || "your_table_id";
  const INPUT_PATH = process.env.INPUT_PATH || "path/to/file_or_folder";

  // Initialize uploader
  const uploader = new DocumentUploader(PROJECT_ID, PAT);

  // Process input
  if (fs.statSync(INPUT_PATH).isFile()) {
    // Single file upload
    console.log("Processing single file...\n");
    const success = await uploader.uploadDocument(INPUT_PATH, TABLE_ID);

    if (success) {
      console.log("\nFile uploaded successfully!");
    } else {
      console.log("\nFile upload failed.");
    }
  } else if (fs.statSync(INPUT_PATH).isDirectory()) {
    // Batch folder upload
    console.log("Processing folder...\n");
    const { successful, failed } = await processFolder(
      INPUT_PATH,
      uploader,
      TABLE_ID
    );

    console.log("\n=== Upload Summary ===");
    console.log(`Successful: ${successful.length}`);
    console.log(`Failed: ${failed.length}`);

    if (successful.length > 0) {
      console.log("\nSuccessful uploads:");
      successful.forEach((file) => console.log(`  ✓ ${file}`));
    }

    if (failed.length > 0) {
      console.log("\nFailed uploads:");
      failed.forEach((file) => console.log(`  ✗ ${file}`));
    }
  } else {
    console.log("Invalid input path");
  }
}

main().catch(console.error);
```

## 6. Usage Examples

### 6.1 Single File Upload

```typescript
// Initialize uploader
const uploader = new DocumentUploader(PROJECT_ID, PAT);

// Upload single document
const success = await uploader.uploadDocument(
  "path/to/document.pdf",
  "your_table_id"
);

if (success) {
  console.log("Upload successful!");
}
```

### 6.2 Batch Folder Upload

```typescript
const { successful, failed } = await processFolder(
  "path/to/documents/folder",
  uploader,
  "your_table_id"
);

console.log(`Successfully uploaded: ${successful.length} files`);
console.log(`Failed uploads: ${failed.length} files`);
```

### 6.3 Custom Chunk Settings

```typescript
// Upload with custom chunk size and overlap
await uploader.uploadDocument(
  "path/to/document.pdf",
  "your_table_id",
  2000, // Custom chunk size
  400 // Custom chunk overlap
);
```

## 7. Best Practices

### 7.1 Chunking Strategy

**Recommended Settings by Document Type:**

| Document Type | Chunk Size | Overlap | Reasoning                   |
| ------------- | ---------- | ------- | --------------------------- |
| PDFs          | 1500       | 300     | Preserves paragraph context |
| Text Files    | 1000       | 200     | Balanced for general text   |
| Spreadsheets  | 800        | 150     | Smaller for structured data |
| JSON/JSONL    | 800        | 150     | Maintains object boundaries |

### 7.2 Error Handling

```typescript
async function robustUpload(
  filePath: string,
  tableId: string,
  maxRetries: number = 3
) {
  const uploader = new DocumentUploader(PROJECT_ID, PAT);

  for (let i = 0; i < maxRetries; i++) {
    try {
      const success = await uploader.uploadDocument(filePath, tableId);
      if (success) return true;
    } catch (error) {
      console.log(`Attempt ${i + 1} failed: ${error}`);
      if (i === maxRetries - 1) throw error;
      await new Promise((resolve) => setTimeout(resolve, 1000 * (i + 1)));
    }
  }
  return false;
}
```

### 7.3 Batch Processing with Rate Limiting

```typescript
async function batchUploadWithLimit(
  files: string[],
  tableId: string,
  concurrentLimit: number = 3
) {
  const uploader = new DocumentUploader(PROJECT_ID, PAT);
  const results = [];

  for (let i = 0; i < files.length; i += concurrentLimit) {
    const batch = files.slice(i, i + concurrentLimit);
    const batchResults = await Promise.all(
      batch.map((file) => uploader.uploadDocument(file, tableId))
    );
    results.push(...batchResults);

    // Rate limiting delay
    if (i + concurrentLimit < files.length) {
      await new Promise((resolve) => setTimeout(resolve, 1000));
    }
  }

  return results;
}
```

## 8. Running the Script

### Using Environment Variables

1. Create `.env` file:

   ```
   JAMAI_PROJECT_ID=your_project_id
   JAMAI_API_KEY=your_PAT
   TABLE_ID=your_table_id
   INPUT_PATH=path/to/documents
   ```
2. Run the script:

   ```bash
   ts-node knowledge_uploader.ts
   ```

### Using Command Line (modify script to accept args)

```bash
ts-node knowledge_uploader.ts \
    --project-id your_project_id \
    --pat your_PAT \
    --table-id your_table_id \
    --input path/to/documents
```

## 9. Troubleshooting

### Common Issues

1. **File not found error**
   * Verify file path is correct
   * Use absolute paths for reliability
2. **Unsupported file format**
   * Check file extension against supported types
   * Verify file is not corrupted
3. **Upload timeout**
   * Check internet connection
   * For large files, consider increasing timeout settings
   * Verify file size is reasonable
4. **Chunk size errors**
   * Ensure chunk size > chunk overlap
   * Try default settings first
   * Adjust based on document structure

## 10. Next Steps

{% stepper %}
{% step %}

### Query Your Knowledge Base

* Use hybrid search to find relevant documents.
* Combine with Action Tables for RAG applications.
  {% endstep %}

{% step %}

### Create RAG Applications

* Link Knowledge Tables to Action Tables.
* Build question-answering systems.
* Develop intelligent document assistants.
  {% endstep %}

{% step %}

### Monitor and Optimize

* Review search quality.
* Adjust chunk settings if needed.
* Add more documents to improve coverage.
  {% endstep %}
  {% endstepper %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.jamaibase.com/developer-reference/typescript-sdk-documentation/quick-start-knowledge-table.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.