feat: Phase 2 — document upload from KB UI
Adds a "ניהול" tab with multipart file upload, async job tracking, and a recent-jobs list with live status polling. Browser POSTs to /KnowledgeBase/action/upload (PHP proxy), which forwards as multipart to shira-hermes /admin/kb/upload. shira-hermes returns a job_id immediately and processes parse/chunk/embed in a background task; the browser polls every 2s until status hits done|failed. New EspoCRM endpoints: POST /KnowledgeBase/action/upload (multipart, $_FILES['file']) GET /KnowledgeBase/action/jobs (list, filtered by topicId/status) GET /KnowledgeBase/action/job?id (single-job detail) The X-User-Name header is forwarded so kb_ingest_job records who uploaded each file. Cross-topic guard in switchTopic stops polling when the user changes topics mid-upload (the job continues server-side). Depends on shira-hermes commit 0cb89fb (admin upload endpoints + migration 002). Refs Task Master #13 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -137,20 +137,20 @@
|
||||
"id": "12",
|
||||
"title": "Phase 1 — Multi-topic KB foundation (DB schema, topic-aware endpoints, topic selector)",
|
||||
"description": "Generalize the KB from being insurance-only to supporting multiple legal domains within one CRM (e.g. ביטוח לאומי, דיני עבודה, דין פלילי). A 'topic' is the firm-level grouping; each kb_source belongs to exactly one topic. Search/ask are scoped to a single topic at a time, chosen from a topic dropdown the user controls. The system_prompt for /kb/ask becomes per-topic so Shira's domain context is correct (today the prompt hardcodes 'Israeli National Insurance'). Backwards-compatible migration: all 6 existing sources move to topic_id=1 named 'ביטוח לאומי' with the existing system-prompt addendum, and the UI defaults to that topic so nothing visible changes for existing users until they choose another topic.",
|
||||
"status": "in-progress",
|
||||
"status": "done",
|
||||
"priority": "high",
|
||||
"details": "DB migration (shira-hermes side, against insurance_kb DB):\n- New table kb_topic: id PK, slug TEXT UNIQUE, name TEXT NOT NULL, description TEXT, system_prompt_addendum TEXT, is_active BOOLEAN DEFAULT true, created_at TIMESTAMPTZ DEFAULT now(), updated_at TIMESTAMPTZ DEFAULT now().\n- Seed: INSERT (id=1, slug='national-insurance', name='ביטוח לאומי', system_prompt_addendum=<copy from current /kb/ask hardcoded text>, is_active=true). RENAME the DB itself? Probably leave as 'insurance_kb' — costly to rename, no real value. Document in memory.\n- ALTER TABLE kb_source ADD COLUMN topic_id INTEGER REFERENCES kb_topic(id). Backfill: UPDATE kb_source SET topic_id = 1. Then ALTER COLUMN SET NOT NULL.\n\nshira-hermes endpoints:\n- New GET /kb/topics — public, returns active topics with id, slug, name, description (no prompt).\n- /kb/search: accepts optional topic_id (single int). When present, the SQL adds AND s.topic_id = $N. When absent (back-compat), still returns all (might want to deprecate this and require topic_id later).\n- /kb/sources, /kb/source/{id}/chunks, /kb/source/{id}/pdf: same — optional topic filter on listings, no change to single-source endpoints.\n- /kb/ask + /kb/ask/stream: accept topic_id param; load topic.system_prompt_addendum and inject into system_prompt instead of the hardcoded insurance text in _build_ask_runner_context. Pin the legal_kb tool to topic_id too (so search_insurance_kb queries are constrained — RENAME the tool to search_legal_kb and pass topic_id at registration time).\n- _expand_query in kb_search.py: parametrize the prompt by topic name (currently hardcoded 'הביטוח הלאומי בישראל'). Pass topic.name in.\n\nKnowledgeBase extension (EspoCRM):\n- New route+action GET /KnowledgeBase/action/topics → proxies /kb/topics.\n- Service.search/ask gain topic_id, controller passes through.\n- Template gets a topic <select> at the top (above the tabs); fetches /KnowledgeBase/action/topics on mount, persists selection to localStorage 'kb-topic'. Default: topic id=1 if it exists, else first active topic.\n- this.kind etc. already pass through; add this.topicId. Pass topic_id with every search/ask call.\n\nConsequences for sessionStorage replay (v0.1.9, v0.1.10): the cached _lastAsk / _lastSearch should record topic_id; on remount only replay if the current topic matches. Else show empty state.",
|
||||
"testStrategy": "1) After migration: SELECT count(*) FROM kb_source WHERE topic_id IS NULL → 0; SELECT name FROM kb_topic → 'ביטוח לאומי'. 2) GET /kb/topics returns the seeded topic. 3) Search/ask with topic_id=1 returns the same 6 sources as before. 4) Add a second topic (manually for the test) and a single test source under it; verify topic_id=1 search no longer surfaces the test source and vice versa. 5) /kb/ask system prompt confirmed via logs to include the topic-specific addendum, not the hardcoded text.",
|
||||
"subtasks": [],
|
||||
"dependencies": [],
|
||||
"createdAt": "2026-04-25T13:30:00Z",
|
||||
"updatedAt": "2026-04-25T14:15:58.498Z"
|
||||
"updatedAt": "2026-04-25T14:34:08.086Z"
|
||||
},
|
||||
{
|
||||
"id": "13",
|
||||
"title": "Phase 2 — Document upload from the KB UI (file picker + async ingestion + job tracking)",
|
||||
"description": "Today documents enter the KB by manually putting them in the MinIO inbox/ folder and waiting for the n8n cron to call /admin/kb/scan-inbox. End users have no path to upload via the UI. Add a file-upload form in the management panel: user picks a PDF/DOCX/TXT, picks a kind (law/regulation/circular), confirms the topic, optionally fills metadata (title, identifier, dates, source_url), and clicks upload. The file lands in MinIO under inbox/<topic_slug>/<kind>/, a kb_ingest_job row is created, and a background asyncio task processes it (parse → chunk → embed → upsert kb_source). The UI polls the job status and shows progress live; on done it links to the new source in the management table.",
|
||||
"status": "pending",
|
||||
"status": "in-progress",
|
||||
"priority": "high",
|
||||
"details": "DB:\n- New table kb_ingest_job: id PK, source_id INTEGER NULL FK kb_source(id) ON DELETE SET NULL, original_filename TEXT, s3_key TEXT, kind TEXT, topic_id INTEGER NOT NULL, metadata_json JSONB, status TEXT NOT NULL CHECK (status IN ('queued','processing','done','failed')), error_message TEXT, chunks_created INTEGER, created_at TIMESTAMPTZ DEFAULT now(), started_at TIMESTAMPTZ, completed_at TIMESTAMPTZ, requested_by_user TEXT.\n- Index: (status, created_at DESC) for the queued list.\n\nshira-hermes endpoints:\n- POST /admin/kb/upload (multipart): receives file, kind, topic_id, optional title/identifier/published_at/effective_at/source_url. Validates topic exists. Writes file to s3 inbox/<topic_slug>/<kind>/<uuid-prefixed filename>. Inserts kb_ingest_job with status=queued. Schedules asyncio.create_task(_process_ingest_job(job_id)) so processing starts immediately, not on the next n8n cron tick. Returns {job_id, status:'queued'}.\n- GET /admin/kb/jobs?topic_id=&status=&limit=: list jobs, newest first.\n- GET /admin/kb/jobs/{id}: single job detail.\n- _process_ingest_job(job_id): UPDATE status='processing', started_at=now. Calls existing kb_ingest.ingest_source. On success: UPDATE status='done', source_id=<new>, chunks_created=<n>, completed_at=now. On failure: UPDATE status='failed', error_message=<exception text>. The upload doesn't block the HTTP request — the user gets the job_id immediately and polls.\n- Reuses scan-inbox advisory lock pattern so a manually-uploaded file plus an n8n cron run don't double-process.\n\nKnowledgeBase extension:\n- EspoCRM PHP proxy for multipart upload — Controllers can read $request->getParsedBody() but multipart needs special handling. Look at how other extensions do it (LegalAssistance/DigitalSignature might have examples).\n- New tab 'ניהול' (visible only to non-portal users; can also gate to admin role later). The tab opens a panel with: (a) Upload form, (b) Sources table for current topic [implemented in Phase 3], (c) Recent jobs.\n- Upload form fields: file picker, kind dropdown, topic dropdown (defaulting to currently-selected topic), optional title/identifier/dates/source_url collapse. Submit → POST KnowledgeBase/action/upload → returns job_id → switch to job-status view that polls every 2s.\n- Job status view: shows the job's status badge (queued/processing/done/failed), elapsed time, and on done: link to the source in the management table (Phase 3).\n\nFile size: enforce 50MB limit at the FastAPI route + server-side. PDFs of עשרות-שעות could approach this.",
|
||||
"testStrategy": "1) Upload a small test PDF (~1MB). Job goes queued → processing → done in <30s. UI shows progress live. 2) Upload a corrupt file. Job ends in failed with a useful error_message. 3) Upload while another large ingest is running — job shows 'processing' but doesn't deadlock. 4) Refresh the page mid-upload — job status restored from sessionStorage + a fresh poll. 5) /admin/kb/jobs returns the upload's row.",
|
||||
@@ -158,7 +158,8 @@
|
||||
"dependencies": [
|
||||
"12"
|
||||
],
|
||||
"createdAt": "2026-04-25T13:30:00Z"
|
||||
"createdAt": "2026-04-25T13:30:00Z",
|
||||
"updatedAt": "2026-04-25T16:20:11.931Z"
|
||||
},
|
||||
{
|
||||
"id": "14",
|
||||
@@ -218,9 +219,9 @@
|
||||
],
|
||||
"metadata": {
|
||||
"version": "1.0.0",
|
||||
"lastModified": "2026-04-25T14:15:58.500Z",
|
||||
"lastModified": "2026-04-25T16:20:11.932Z",
|
||||
"taskCount": 17,
|
||||
"completedCount": 8,
|
||||
"completedCount": 9,
|
||||
"tags": [
|
||||
"master"
|
||||
]
|
||||
|
||||
@@ -20,6 +20,9 @@
|
||||
<li class="{{#ifEqual mode 'browse'}}active{{/ifEqual}}">
|
||||
<a href="#" role="button" data-action="switchMode" data-mode="browse">עיון</a>
|
||||
</li>
|
||||
<li class="{{#ifEqual mode 'manage'}}active{{/ifEqual}}">
|
||||
<a href="#" role="button" data-action="switchMode" data-mode="manage">ניהול</a>
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
{{#ifEqual mode 'search'}}
|
||||
@@ -66,5 +69,73 @@
|
||||
</div>
|
||||
{{/ifEqual}}
|
||||
|
||||
{{#ifEqual mode 'manage'}}
|
||||
<div class="kb-manage" style="display:flex;flex-direction:column;gap:16px;">
|
||||
<div class="panel panel-default" style="padding:12px;">
|
||||
<h4 style="margin-top:0;">העלאת מסמך חדש</h4>
|
||||
<form class="kb-upload-form" enctype="multipart/form-data"
|
||||
style="display:flex;flex-direction:column;gap:10px;">
|
||||
<div style="display:flex;gap:8px;align-items:center;flex-wrap:wrap;">
|
||||
<label for="kb-upload-file" class="text-muted" style="margin:0;font-weight:normal;min-width:80px;">קובץ:</label>
|
||||
<input type="file" id="kb-upload-file" name="file"
|
||||
accept=".pdf,.docx,.txt"
|
||||
style="flex:1 1 280px;" />
|
||||
</div>
|
||||
<div style="display:flex;gap:8px;align-items:center;flex-wrap:wrap;">
|
||||
<label for="kb-upload-kind" class="text-muted" style="margin:0;font-weight:normal;min-width:80px;">סוג:</label>
|
||||
<select id="kb-upload-kind" class="form-control" name="kind"
|
||||
style="flex:0 0 auto;width:200px;">
|
||||
<option value="law">חוק</option>
|
||||
<option value="regulation">תקנות</option>
|
||||
<option value="circular">חוזר</option>
|
||||
</select>
|
||||
</div>
|
||||
<div style="display:flex;gap:8px;align-items:center;flex-wrap:wrap;">
|
||||
<label for="kb-upload-title" class="text-muted" style="margin:0;font-weight:normal;min-width:80px;">כותרת:</label>
|
||||
<input type="text" id="kb-upload-title" class="form-control" name="title"
|
||||
placeholder="ברירת מחדל: שם הקובץ"
|
||||
style="flex:1 1 280px;" />
|
||||
</div>
|
||||
<details>
|
||||
<summary class="text-muted" style="cursor:pointer;">מטא-דאטה נוסף (אופציונלי)</summary>
|
||||
<div style="display:flex;flex-direction:column;gap:8px;margin-top:8px;padding-right:12px;">
|
||||
<div style="display:flex;gap:8px;align-items:center;flex-wrap:wrap;">
|
||||
<label class="text-muted" style="margin:0;font-weight:normal;min-width:80px;">מזהה:</label>
|
||||
<input type="text" class="form-control" name="identifier"
|
||||
placeholder="למשל: ח'(353) 14.1.2018"
|
||||
style="flex:1 1 280px;" />
|
||||
</div>
|
||||
<div style="display:flex;gap:8px;align-items:center;flex-wrap:wrap;">
|
||||
<label class="text-muted" style="margin:0;font-weight:normal;min-width:80px;">פורסם ב:</label>
|
||||
<input type="date" class="form-control" name="published_at"
|
||||
style="flex:0 0 auto;width:200px;" />
|
||||
<label class="text-muted" style="margin:0;font-weight:normal;">תוקף מ:</label>
|
||||
<input type="date" class="form-control" name="effective_at"
|
||||
style="flex:0 0 auto;width:200px;" />
|
||||
</div>
|
||||
<div style="display:flex;gap:8px;align-items:center;flex-wrap:wrap;">
|
||||
<label class="text-muted" style="margin:0;font-weight:normal;min-width:80px;">קישור מקור:</label>
|
||||
<input type="url" class="form-control" name="source_url"
|
||||
placeholder="https://…"
|
||||
style="flex:1 1 280px;" />
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
<div style="display:flex;gap:8px;align-items:center;">
|
||||
<button type="button" class="btn btn-primary" data-action="submitUpload">העלה</button>
|
||||
<span class="kb-upload-hint text-muted small">מקסימום 50MB. PDF / DOCX / TXT.</span>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
|
||||
<div class="panel panel-default" style="padding:12px;">
|
||||
<h4 style="margin-top:0;">משימות אחרונות</h4>
|
||||
<div class="kb-jobs-list">
|
||||
<div class="text-muted">טוען…</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{{/ifEqual}}
|
||||
|
||||
<div class="kb-results" style="margin-top:12px;"></div>
|
||||
</div>
|
||||
|
||||
@@ -12,6 +12,9 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
|
||||
let _lastSearch = null; // {query, kind, topicId, hits, selectedIdx, completedAt}
|
||||
let _topics = null; // [{id, slug, name, description, is_active}]
|
||||
let _topicsPromise = null; // dedupe concurrent fetches across mounts
|
||||
let _activeJobId = null; // currently-polled ingest job id (across remounts)
|
||||
let _activeJobTopicId = null; // topic this job belongs to (for cross-topic guard)
|
||||
let _pollIntervalId = null; // setInterval handle so remounts don't double-poll
|
||||
|
||||
const SS_ASK = 'kb-last-ask';
|
||||
const SS_SEARCH = 'kb-last-search';
|
||||
@@ -117,6 +120,10 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
|
||||
if (!Number.isFinite(id) || id === this.topicId) return;
|
||||
this.switchTopic(id);
|
||||
},
|
||||
'click [data-action="submitUpload"]': function (e) {
|
||||
e.preventDefault();
|
||||
this.runUpload();
|
||||
},
|
||||
},
|
||||
|
||||
// Cancel any in-flight ask/search, drop the cached last-result for
|
||||
@@ -190,6 +197,214 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
|
||||
});
|
||||
},
|
||||
|
||||
// Phase 2: lazy-loads the recent-jobs list and resumes polling of any
|
||||
// in-flight upload job. No-op if .kb-jobs-list isn't in the DOM (i.e.
|
||||
// the user isn't currently on the 'manage' tab).
|
||||
_ensureManagePanelLoaded: function () {
|
||||
const self = this;
|
||||
const $list = this.$el.find('.kb-jobs-list');
|
||||
if (!$list.length || this.topicId == null) return;
|
||||
|
||||
// Resume polling if we navigated away mid-upload and came back.
|
||||
if (_activeJobId && _activeJobTopicId === this.topicId) {
|
||||
this._startJobPolling(_activeJobId);
|
||||
} else if (_activeJobId && _activeJobTopicId !== this.topicId) {
|
||||
// Cross-topic guard: an upload in another topic is invisible
|
||||
// here. The polling continues silently in the background.
|
||||
}
|
||||
|
||||
// Always refresh the recent-jobs list on mount.
|
||||
this._loadJobsList();
|
||||
},
|
||||
|
||||
_loadJobsList: function () {
|
||||
const self = this;
|
||||
const $list = this.$el.find('.kb-jobs-list');
|
||||
if (!$list.length) return;
|
||||
const topicAtFetch = this.topicId;
|
||||
Espo.Ajax.getRequest('KnowledgeBase/action/jobs', {
|
||||
topicId: topicAtFetch,
|
||||
limit: 20,
|
||||
}).then(res => {
|
||||
if (self.topicId !== topicAtFetch) return; // user switched
|
||||
self._renderJobsList(res && res.items || []);
|
||||
}).catch(err => {
|
||||
console.error('KB: failed to load jobs', err);
|
||||
$list.html('<div class="text-muted">שגיאה בטעינת רשימת המשימות.</div>');
|
||||
});
|
||||
},
|
||||
|
||||
_renderJobsList: function (items) {
|
||||
const $list = this.$el.find('.kb-jobs-list');
|
||||
if (!$list.length) return;
|
||||
if (!items.length) {
|
||||
$list.html('<div class="text-muted">אין משימות עדיין.</div>');
|
||||
return;
|
||||
}
|
||||
const statusHe = {
|
||||
queued: '<span class="label label-default">בתור</span>',
|
||||
processing: '<span class="label label-info">מעובד…</span>',
|
||||
done: '<span class="label label-success">הושלם</span>',
|
||||
failed: '<span class="label label-danger">נכשל</span>',
|
||||
};
|
||||
const kindHe = {law: 'חוק', regulation: 'תקנות', circular: 'חוזר'};
|
||||
const rows = items.map(j => {
|
||||
const when = j.completed_at || j.started_at || j.created_at || '';
|
||||
const whenShort = String(when).slice(0, 19).replace('T', ' ');
|
||||
const meta = j.metadata_json || {};
|
||||
const title = this.escape(meta.title || j.original_filename || '—');
|
||||
const fname = this.escape(j.original_filename || '');
|
||||
const detail = j.status === 'done'
|
||||
? `<span class="text-muted">${j.chunks_created || 0} קטעים</span>`
|
||||
: (j.status === 'failed'
|
||||
? `<span class="text-danger" title="${this.escape(j.error_message || '')}">${this.escape((j.error_message || '').slice(0, 80))}</span>`
|
||||
: '');
|
||||
return `<div style="display:flex;justify-content:space-between;align-items:center;border-bottom:1px solid #eee;padding:8px 0;gap:12px;">
|
||||
<div style="flex:1 1 auto;min-width:0;">
|
||||
<div style="font-weight:500;">${title}</div>
|
||||
<div class="text-muted small">
|
||||
${kindHe[j.kind] || j.kind} · ${this.escape(whenShort)}${j.requested_by_user ? ' · ' + this.escape(j.requested_by_user) : ''}
|
||||
</div>
|
||||
${detail ? '<div class="small">' + detail + '</div>' : ''}
|
||||
</div>
|
||||
<div>${statusHe[j.status] || this.escape(j.status)}</div>
|
||||
</div>`;
|
||||
}).join('');
|
||||
$list.html(rows);
|
||||
},
|
||||
|
||||
// Live status panel rendered above the upload form during processing.
|
||||
_renderJobStatus: function (job) {
|
||||
const $form = this.$el.find('.kb-upload-form');
|
||||
if (!$form.length) return;
|
||||
// Remove any previous banner so we can replace it.
|
||||
this.$el.find('.kb-upload-status').remove();
|
||||
|
||||
const statusHe = {
|
||||
queued: 'בתור',
|
||||
processing: 'מעובד…',
|
||||
done: 'הושלם בהצלחה',
|
||||
failed: 'נכשל',
|
||||
};
|
||||
const klass = job.status === 'done' ? 'alert-success'
|
||||
: job.status === 'failed' ? 'alert-danger'
|
||||
: 'alert-info';
|
||||
const filename = this.escape(job.original_filename || '');
|
||||
const extra = job.status === 'done'
|
||||
? ` · נוצרו ${job.chunks_created || 0} קטעים`
|
||||
: (job.status === 'failed'
|
||||
? ` · ${this.escape((job.error_message || '').slice(0, 200))}`
|
||||
: '');
|
||||
const html = `<div class="kb-upload-status alert ${klass}" style="margin-bottom:12px;">
|
||||
<strong>${this.escape(filename)}</strong> — ${statusHe[job.status] || this.escape(job.status)}${extra}
|
||||
</div>`;
|
||||
$form.before(html);
|
||||
},
|
||||
|
||||
_startJobPolling: function (jobId) {
|
||||
const self = this;
|
||||
this._stopJobPolling();
|
||||
_activeJobId = jobId;
|
||||
_activeJobTopicId = this.topicId;
|
||||
|
||||
const tick = () => {
|
||||
Espo.Ajax.getRequest('KnowledgeBase/action/job', {id: jobId})
|
||||
.then(job => {
|
||||
if (!self.$el || !self.$el.length) return;
|
||||
self._renderJobStatus(job);
|
||||
if (job.status === 'done' || job.status === 'failed') {
|
||||
self._stopJobPolling();
|
||||
_activeJobId = null;
|
||||
_activeJobTopicId = null;
|
||||
// Refresh the recent-jobs list to surface the new row.
|
||||
self._loadJobsList();
|
||||
}
|
||||
})
|
||||
.catch(err => {
|
||||
console.error('KB: job poll failed', err);
|
||||
// Don't kill the interval on a single transient error;
|
||||
// let the next tick try again.
|
||||
});
|
||||
};
|
||||
tick(); // immediate first read so the user sees status fast
|
||||
_pollIntervalId = setInterval(tick, 2000);
|
||||
},
|
||||
|
||||
_stopJobPolling: function () {
|
||||
if (_pollIntervalId) {
|
||||
clearInterval(_pollIntervalId);
|
||||
_pollIntervalId = null;
|
||||
}
|
||||
},
|
||||
|
||||
runUpload: function () {
|
||||
const $form = this.$el.find('.kb-upload-form');
|
||||
const $file = $form.find('input[type="file"]');
|
||||
const file = ($file[0] && $file[0].files && $file[0].files[0]) || null;
|
||||
if (!file) {
|
||||
alert('יש לבחור קובץ.');
|
||||
return;
|
||||
}
|
||||
if (file.size > 50 * 1024 * 1024) {
|
||||
alert('הקובץ גדול מ-50MB. אנא העלה קובץ קטן יותר.');
|
||||
return;
|
||||
}
|
||||
if (this.topicId == null) {
|
||||
alert('נושא לא נבחר.');
|
||||
return;
|
||||
}
|
||||
|
||||
const fd = new FormData();
|
||||
fd.append('file', file, file.name);
|
||||
fd.append('kind', $form.find('select[name="kind"]').val() || 'circular');
|
||||
fd.append('topicId', String(this.topicId));
|
||||
['title', 'identifier', 'published_at', 'effective_at', 'source_url'].forEach(name => {
|
||||
const v = ($form.find(`[name="${name}"]`).val() || '').trim();
|
||||
if (v) fd.append(name, v);
|
||||
});
|
||||
|
||||
const $btn = this.$el.find('[data-action="submitUpload"]');
|
||||
$btn.prop('disabled', true).text('מעלה…');
|
||||
|
||||
const self = this;
|
||||
// Bypass Espo.Ajax.postRequest (which JSON-encodes the body) — use
|
||||
// raw $.ajax so the FormData multipart body is sent intact.
|
||||
$.ajax({
|
||||
url: 'api/v1/KnowledgeBase/action/upload',
|
||||
method: 'POST',
|
||||
data: fd,
|
||||
contentType: false,
|
||||
processData: false,
|
||||
cache: false,
|
||||
}).then(res => {
|
||||
$btn.prop('disabled', false).text('העלה');
|
||||
// Reset the form so the user can upload another.
|
||||
$form[0].reset();
|
||||
if (res && res.job_id) {
|
||||
self._renderJobStatus({
|
||||
original_filename: file.name,
|
||||
status: res.status || 'queued',
|
||||
});
|
||||
self._startJobPolling(res.job_id);
|
||||
}
|
||||
}).fail(xhr => {
|
||||
$btn.prop('disabled', false).text('העלה');
|
||||
let msg = 'שגיאה לא ידועה';
|
||||
try {
|
||||
const json = JSON.parse(xhr.responseText || '{}');
|
||||
msg = json.message || json.detail || xhr.responseText || msg;
|
||||
} catch (e) {
|
||||
msg = xhr.responseText || msg;
|
||||
}
|
||||
self.$el.find('.kb-upload-status').remove();
|
||||
self.$el.find('.kb-upload-form').before(
|
||||
'<div class="kb-upload-status alert alert-danger" style="margin-bottom:12px;">'
|
||||
+ 'העלאה נכשלה: ' + self.escape(msg.slice(0, 300))
|
||||
+ '</div>'
|
||||
);
|
||||
});
|
||||
},
|
||||
|
||||
_ensureTopicsLoaded: function () {
|
||||
if (_topics) return Promise.resolve(_topics);
|
||||
if (_topicsPromise) return _topicsPromise;
|
||||
@@ -253,6 +468,13 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
|
||||
_clear(sessionStorage, SS_SEARCH);
|
||||
this._sources = null;
|
||||
this._sourcesPromise = null;
|
||||
// An in-flight upload belongs to the topic where it started;
|
||||
// stop polling it here so the new topic doesn't see a stale banner.
|
||||
// The job continues processing on the server; the user can find
|
||||
// it under "משימות אחרונות" if they switch back.
|
||||
this._stopJobPolling();
|
||||
_activeJobId = null;
|
||||
_activeJobTopicId = null;
|
||||
this.stopAskProgress();
|
||||
this.setLoading(false);
|
||||
this.$el.find('.kb-results').empty();
|
||||
@@ -272,6 +494,7 @@ define('modules/knowledge-base/views/kb/index', ['view'], function (Dep) {
|
||||
if (!self.$el || !self.$el.length) return;
|
||||
self._populateTopicPicker();
|
||||
self._ensureSourcesLoaded();
|
||||
self._ensureManagePanelLoaded();
|
||||
};
|
||||
if (_topics) {
|
||||
afterTopics();
|
||||
|
||||
@@ -121,4 +121,74 @@ class KnowledgeBase
|
||||
$this->coerceTopicId($data->topicId ?? $data->topic_id ?? null)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Multipart upload to shira-hermes /admin/kb/upload. We rely on PHP's
|
||||
* automatic multipart parsing via $_FILES because the EspoCRM Request
|
||||
* interface doesn't expose uploaded files directly. The user's
|
||||
* EspoCRM identity is forwarded as `X-User-Name` so the upstream job
|
||||
* row records who uploaded what.
|
||||
*/
|
||||
public function postActionUpload(Request $request, Response $response): array
|
||||
{
|
||||
$this->checkAccess();
|
||||
|
||||
$files = $_FILES['file'] ?? null;
|
||||
if (!$files || ($files['error'] ?? UPLOAD_ERR_NO_FILE) !== UPLOAD_ERR_OK) {
|
||||
throw new BadRequest('No file uploaded (form field must be `file`).');
|
||||
}
|
||||
$tmpPath = $files['tmp_name'] ?? '';
|
||||
if (!$tmpPath || !is_uploaded_file($tmpPath)) {
|
||||
throw new BadRequest('Invalid upload — tmp_name missing.');
|
||||
}
|
||||
|
||||
// For multipart/form-data EspoCRM's getParsedBody() returns an empty
|
||||
// stdClass; the form fields land in $_POST as PHP parses them.
|
||||
$kind = $_POST['kind'] ?? null;
|
||||
if (!in_array($kind, ['law', 'regulation', 'circular'], true)) {
|
||||
throw new BadRequest('kind must be one of: law, regulation, circular.');
|
||||
}
|
||||
$topicId = $this->coerceTopicId($_POST['topicId'] ?? $_POST['topic_id'] ?? null);
|
||||
if ($topicId === null) {
|
||||
throw new BadRequest('topicId is required.');
|
||||
}
|
||||
|
||||
$metadata = [
|
||||
'title' => isset($_POST['title']) ? trim((string) $_POST['title']) : '',
|
||||
'identifier' => isset($_POST['identifier']) ? trim((string) $_POST['identifier']) : '',
|
||||
'published_at' => $_POST['published_at'] ?? $_POST['publishedAt'] ?? '',
|
||||
'effective_at' => $_POST['effective_at'] ?? $_POST['effectiveAt'] ?? '',
|
||||
'source_url' => $_POST['source_url'] ?? $_POST['sourceUrl'] ?? '',
|
||||
];
|
||||
|
||||
return $this->getService()->uploadFile(
|
||||
$tmpPath,
|
||||
(string) ($files['name'] ?? 'upload'),
|
||||
(string) $kind,
|
||||
$topicId,
|
||||
$metadata,
|
||||
(string) $this->user->get('userName')
|
||||
);
|
||||
}
|
||||
|
||||
public function getActionJobs(Request $request, Response $response): array
|
||||
{
|
||||
$this->checkAccess();
|
||||
$topicId = $this->coerceTopicId($request->getQueryParam('topicId'));
|
||||
$status = $request->getQueryParam('status');
|
||||
$limit = $request->getQueryParam('limit');
|
||||
$limitInt = ($limit !== null && is_numeric($limit)) ? (int) $limit : 50;
|
||||
|
||||
return $this->getService()->listJobs($topicId, $status, $limitInt);
|
||||
}
|
||||
|
||||
public function getActionJob(Request $request, Response $response): array
|
||||
{
|
||||
$this->checkAccess();
|
||||
$jobId = $request->getQueryParam('id');
|
||||
if (!$jobId || !is_numeric($jobId)) {
|
||||
throw new BadRequest('id (numeric) is required.');
|
||||
}
|
||||
return $this->getService()->getJob((int) $jobId);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -38,5 +38,29 @@
|
||||
"controller": "KnowledgeBase",
|
||||
"action": "ask"
|
||||
}
|
||||
},
|
||||
{
|
||||
"route": "/KnowledgeBase/action/upload",
|
||||
"method": "post",
|
||||
"params": {
|
||||
"controller": "KnowledgeBase",
|
||||
"action": "upload"
|
||||
}
|
||||
},
|
||||
{
|
||||
"route": "/KnowledgeBase/action/jobs",
|
||||
"method": "get",
|
||||
"params": {
|
||||
"controller": "KnowledgeBase",
|
||||
"action": "jobs"
|
||||
}
|
||||
},
|
||||
{
|
||||
"route": "/KnowledgeBase/action/job",
|
||||
"method": "get",
|
||||
"params": {
|
||||
"controller": "KnowledgeBase",
|
||||
"action": "job"
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
@@ -70,6 +70,136 @@ class KnowledgeBaseService
|
||||
return $this->get($path);
|
||||
}
|
||||
|
||||
/**
|
||||
* Phase 2: async upload to /admin/kb/upload. Forwards the temp-uploaded
|
||||
* file via cURL multipart and the EspoCRM username as `X-User-Name`
|
||||
* so shira-hermes can record who uploaded what.
|
||||
*
|
||||
* @return array{job_id:int, status:string}
|
||||
*/
|
||||
public function uploadFile(
|
||||
string $tmpPath,
|
||||
string $filename,
|
||||
string $kind,
|
||||
int $topicId,
|
||||
array $metadata,
|
||||
string $username
|
||||
): array {
|
||||
if (!is_readable($tmpPath)) {
|
||||
throw new Error('Uploaded file is not readable on disk.');
|
||||
}
|
||||
if (!in_array($kind, ['law', 'regulation', 'circular'], true)) {
|
||||
throw new BadRequest('kind must be one of: law, regulation, circular.');
|
||||
}
|
||||
|
||||
$url = $this->getBaseUrl() . '/admin/kb/upload';
|
||||
$apiKey = $this->getApiKey();
|
||||
if (!$apiKey) {
|
||||
throw new Error('SmartAssistant API key is not configured.');
|
||||
}
|
||||
|
||||
// Build the multipart form. CURLOPT_POSTFIELDS as an array makes
|
||||
// cURL set the multipart Content-Type with a generated boundary.
|
||||
$fields = [
|
||||
'kind' => $kind,
|
||||
'topic_id' => (string) $topicId,
|
||||
'file' => new \CURLFile($tmpPath, $this->guessMime($filename), $filename),
|
||||
];
|
||||
foreach (['title', 'identifier', 'published_at', 'effective_at', 'source_url'] as $k) {
|
||||
$val = $metadata[$k] ?? '';
|
||||
if ($val !== '' && $val !== null) {
|
||||
$fields[$k] = (string) $val;
|
||||
}
|
||||
}
|
||||
|
||||
$ch = curl_init($url);
|
||||
curl_setopt_array($ch, [
|
||||
CURLOPT_POST => true,
|
||||
CURLOPT_POSTFIELDS => $fields,
|
||||
CURLOPT_HTTPHEADER => [
|
||||
'Accept: application/json',
|
||||
'X-Api-Key: ' . $apiKey,
|
||||
'X-User-Name: ' . $username,
|
||||
],
|
||||
CURLOPT_RETURNTRANSFER => true,
|
||||
CURLOPT_TIMEOUT => 120,
|
||||
CURLOPT_CONNECTTIMEOUT => 10,
|
||||
]);
|
||||
|
||||
$body = curl_exec($ch);
|
||||
$httpCode = (int) curl_getinfo($ch, CURLINFO_HTTP_CODE);
|
||||
$err = curl_error($ch);
|
||||
curl_close($ch);
|
||||
|
||||
if ($err) {
|
||||
$this->log->error("KnowledgeBase: upload transport error: {$err}");
|
||||
throw new Error("Failed to reach Knowledge Base: {$err}");
|
||||
}
|
||||
if ($httpCode === 413) {
|
||||
throw new BadRequest('File too large (max 50MB).');
|
||||
}
|
||||
if ($httpCode >= 400 && $httpCode < 500) {
|
||||
$detail = $this->decodeDetail($body) ?: 'Bad request';
|
||||
throw new BadRequest($detail);
|
||||
}
|
||||
if ($httpCode < 200 || $httpCode >= 300) {
|
||||
$this->log->error("KnowledgeBase: upload HTTP {$httpCode}: {$body}");
|
||||
throw new Error("Knowledge Base returned HTTP {$httpCode}");
|
||||
}
|
||||
|
||||
$decoded = json_decode((string) $body, true);
|
||||
if (!is_array($decoded) || !isset($decoded['job_id'])) {
|
||||
throw new Error('Invalid response from Knowledge Base.');
|
||||
}
|
||||
return $decoded;
|
||||
}
|
||||
|
||||
public function listJobs(?int $topicId, ?string $status, int $limit): array
|
||||
{
|
||||
$limit = max(1, min(200, $limit));
|
||||
$qs = ['limit' => $limit];
|
||||
if ($topicId !== null) {
|
||||
$qs['topic_id'] = $topicId;
|
||||
}
|
||||
if ($status !== null && $status !== '') {
|
||||
$qs['status'] = $status;
|
||||
}
|
||||
$path = '/admin/kb/jobs?' . http_build_query($qs);
|
||||
return $this->get($path);
|
||||
}
|
||||
|
||||
public function getJob(int $jobId): array
|
||||
{
|
||||
return $this->get('/admin/kb/jobs/' . $jobId);
|
||||
}
|
||||
|
||||
private function guessMime(string $filename): string
|
||||
{
|
||||
$lower = strtolower($filename);
|
||||
if (str_ends_with($lower, '.pdf')) {
|
||||
return 'application/pdf';
|
||||
}
|
||||
if (str_ends_with($lower, '.docx')) {
|
||||
return 'application/vnd.openxmlformats-officedocument.wordprocessingml.document';
|
||||
}
|
||||
if (str_ends_with($lower, '.txt')) {
|
||||
return 'text/plain';
|
||||
}
|
||||
return 'application/octet-stream';
|
||||
}
|
||||
|
||||
private function decodeDetail($body): ?string
|
||||
{
|
||||
if (!is_string($body)) {
|
||||
return null;
|
||||
}
|
||||
$decoded = json_decode($body, true);
|
||||
if (is_array($decoded) && isset($decoded['detail'])) {
|
||||
return is_string($decoded['detail']) ? $decoded['detail'] : json_encode($decoded['detail']);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
public function ask(
|
||||
string $message,
|
||||
?string $conversationId,
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "KnowledgeBase",
|
||||
"module": "KnowledgeBase",
|
||||
"version": "0.3.0",
|
||||
"version": "0.4.0",
|
||||
"acceptableVersions": [
|
||||
">=8.0.0"
|
||||
],
|
||||
|
||||
Reference in New Issue
Block a user