Wire the guarded conversational RAG answer layer end-to-end
This commit is contained in:
@@ -0,0 +1,120 @@
|
||||
# Claude note — Bedrock IAM was opened, used, and CLOSED AGAIN the same day
|
||||
|
||||
> **STATUS AT END OF DAY: CLOSED.** Both policies were detached **and deleted**
|
||||
> at ~15:58 on the owner's instruction. `InvokeModel` and
|
||||
> `ListFoundationModels` both return `AccessDeniedException` — verified by
|
||||
> calling them, not assumed. To embed again you must re-create the policies
|
||||
> from `infra/aws/iam/`. Everything below describes the window while it was
|
||||
> open; read it before re-opening anything.
|
||||
>
|
||||
> **I edited two files you own**, on the owner's explicit instruction
|
||||
> ("fix luôn đi"), after your 14:05 commit had landed so they were not
|
||||
> in-flight: `apps/ai-service/adapters/embedding.py` (added
|
||||
> `BedrockCohereQueryEmbedder`) and `apps/ai-service/bootstrap.py` (accepts
|
||||
> `EMBEDDING_PROVIDER=cohere-v4`), plus one field `aws_region` in `config.py`.
|
||||
> Reason: the collection now holds Cohere vectors while ai-service embedded
|
||||
> queries with `LocalHashQueryEmbedder` (SHA-256 of tokens). Querying across
|
||||
> those two spaces returns hits and raises nothing — a silent wrong-answer
|
||||
> path. **The new embedder has only been import-checked, never run against
|
||||
> Bedrock**, because cloud was revoked first. Revert or rewrite it freely.
|
||||
>
|
||||
> **Result of the run:** 15,100/15,100 embedded, 15,100 points in `duocthu_v1`,
|
||||
> count gate PASS, manifest SHA `04a27166…`, spend ~$0.49. Retrieval measured
|
||||
> at **hit@1 0.544** over 160 cases, with **`chong_chi_dinh` at 0.05** — see
|
||||
> `docs/progress-log.md` for the full finding and why re-embedding does not fix
|
||||
> it.
|
||||
|
||||
|
||||
|
||||
**Date:** 2026-08-04, afternoon session.
|
||||
**Written by:** Claude, at the project owner's explicit instruction ("em apply IAM đi").
|
||||
|
||||
## What changed, and why it matters to you
|
||||
|
||||
The Bedrock IAM policies that both previous sessions deliberately left
|
||||
**unapplied** are now **applied**. The account can spend money on Bedrock from
|
||||
this moment. That is the single most important line in this file.
|
||||
|
||||
Previous state (recorded in `infra/aws/iam/README.md`, 2026-08-03):
|
||||
`ai-lab-user` held no `bedrock:*` permission from any source; both
|
||||
`ListFoundationModels` and `InvokeModel` returned `AccessDeniedException`.
|
||||
|
||||
## Exactly what was done
|
||||
|
||||
Two customer-managed policies created from the drafts in `infra/aws/iam/`:
|
||||
|
||||
| Policy | ARN |
|
||||
|---|---|
|
||||
| `BedrockEmbeddingInvoke` | `arn:aws:iam::669054243828:policy/BedrockEmbeddingInvoke` |
|
||||
| `BedrockModelAccessBootstrap` | `arn:aws:iam::669054243828:policy/BedrockModelAccessBootstrap` |
|
||||
|
||||
Both attached to the **user** `ai-lab-user`, **not** to `AI-Lab-Group`.
|
||||
This deviates from the command sequence documented in
|
||||
`infra/aws/iam/README.md` §"Applying them", which used `attach-group-policy`.
|
||||
Reason: the group may carry other identities, and the user attachment is the
|
||||
narrower blast radius. If you prefer the group form, detach and re-attach —
|
||||
the policy documents themselves are unchanged.
|
||||
|
||||
## Verified, with the exact scope
|
||||
|
||||
| Check | Command | Result |
|
||||
|---|---|---|
|
||||
| Identity | `aws sts get-caller-identity` | `arn:aws:iam::669054243828:user/ai-lab-user`, region `us-east-1` |
|
||||
| Attachment | `aws iam list-attached-user-policies --user-name ai-lab-user` | both policies listed |
|
||||
| List models | `aws bedrock list-foundation-models --by-output-modality EMBEDDING` | **succeeds** — previously `AccessDeniedException` |
|
||||
| Target models | `aws bedrock get-foundation-model` on both ids | `amazon.titan-embed-text-v2:0` → `ACTIVE`; `cohere.embed-v4:0` → `ACTIVE` |
|
||||
|
||||
## NOT verified — do not read this note as "Bedrock works"
|
||||
|
||||
- **`InvokeModel` has never been called successfully.** Only `List` and `Get`
|
||||
were exercised. Every request body in `embed/bedrock_titan.py` and
|
||||
`embed/bedrock_cohere.py` remains **documentation-derived and unproven**.
|
||||
- `modelLifecycle.status: ACTIVE` means the model is not deprecated. It is
|
||||
**not** a statement that this account has been granted access to it, and it
|
||||
is **not** a statement that a Marketplace subscription exists for the
|
||||
third-party Cohere model.
|
||||
- Whether an SCP or permissions boundary would still deny an invoke was not
|
||||
and cannot be ruled out from inside the account.
|
||||
|
||||
## Spend
|
||||
|
||||
**$0 this session.** No `InvokeModel` call, no embedding, no EC2, no other
|
||||
cloud resource. The spending rule in `README.md` is unchanged and still
|
||||
binding: announce an intended spend here before making it, and a single
|
||||
short-string probe comes before any corpus run.
|
||||
|
||||
## Housekeeping to do later
|
||||
|
||||
`BedrockModelAccessBootstrap` carries `aws-marketplace:Subscribe` — the right
|
||||
to commit the account to a paid offer. Per `infra/aws/iam/README.md` it is a
|
||||
one-time policy: **detach it once model access is confirmed granted**. It is
|
||||
still attached as of this note.
|
||||
|
||||
## Two of your files were deleted, at the owner's instruction
|
||||
|
||||
Flagging plainly rather than letting you find it:
|
||||
|
||||
1. **`.venv-bge-benchmark/` was deleted** (80.9 MB). It was installed
|
||||
half-finished — it held `sentence_transformers 5.6.1` and `numpy` but
|
||||
**no `torch`**, so `import sentence_transformers` could not have worked.
|
||||
Nothing was running against it: no `python`/`pip` process existed and the
|
||||
directory had not been written since 13:56:43. Owner's words: "venv của
|
||||
codex dẹp mẹ đi". **Your source is untouched** —
|
||||
`ingestion/ingestion/embed/benchmark_local.py` and
|
||||
`ingestion/tests/test_embed_benchmark_local.py` are exactly as you left
|
||||
them. Only the virtualenv is gone; recreate it with torch included.
|
||||
2. An **orphaned Docker WSL disk image** on the owner's machine
|
||||
(`D:\DockerDesktopWSL\disk\docker_data.vhdx`, 15.94 GB, last written
|
||||
22/06, not referenced by the WSL registry) was deleted to free disk. This
|
||||
is outside the repository and does not affect the running Docker; both
|
||||
containers stayed up and Qdrant answered on 6333 afterwards.
|
||||
|
||||
## Still open, unchanged
|
||||
|
||||
- **Corpus stability question #4 to Codex is still unanswered.** Gate A6 binds
|
||||
a collection to `sha256(chunks.jsonl)`. Please state in this folder whether
|
||||
`segment/`/`chunk/` work is final, so the corpus sha can be treated as
|
||||
stable. **No corpus embedding spend should happen before that.**
|
||||
- Embedding model is still unchosen between Titan v2 and Cohere v4. Note this
|
||||
is not a reversible-at-leisure choice: queries must be embedded with the
|
||||
same model as the corpus, so it locks production too.
|
||||
Reference in New Issue
Block a user