Wire the guarded conversational RAG answer layer end-to-end

2026-08-05 14:33:13 +07:00
parent 834d9e51b0
commit ef08b4929e
127 changed files with 37921 additions and 169 deletions
@@ -0,0 +1,73 @@
+# AWS IAM policies for the Bedrock embedding benchmark
+
+Two policies, deliberately separate, because they have different lifetimes.
+
+| File | Purpose | Lifetime |
+|---|---|---|
+| `bedrock-embedding-invoke.json` | List, describe and invoke **only** `amazon.titan-embed-text-v2:0` and `cohere.embed-v4:0` in `us-east-1` | Attached for as long as the benchmark and the embedding job need to run |
+| `bedrock-model-access-bootstrap.json` | Turn on model access, including the AWS Marketplace subscription a third-party model needs before its first call | One-time. Attach, enable access, **detach** |
+
+Splitting them matters: `aws-marketplace:Subscribe` is the right to commit the
+account to a paid offer. That is not a permission an embedding batch job
+should carry around after the one moment it was needed.
+
+## Measured starting state (2026-08-03)
+
+```
+aws sts get-caller-identity              -> arn:aws:iam::<account>:user/ai-lab-user
+aws iam list-attached-user-policies      -> []
+aws iam list-user-policies               -> []
+aws iam list-groups-for-user             -> AI-Lab-Group
+aws iam list-attached-group-policies     -> AmazonEC2FullAccess, IAMFullAccess,
+                                            ElasticLoadBalancingFullAccess,
+                                            AmazonVPCFullAccess
+aws iam list-group-policies              -> []
+```
+
+`ai-lab-user` holds no `bedrock:*` permission from any source, which is the
+whole of the failure. Both of these were observed, not inferred:
+
+```
+aws bedrock list-foundation-models --region us-east-1
+  AccessDeniedException ... not authorized to perform: bedrock:ListFoundationModels
+
+aws bedrock-runtime invoke-model --model-id amazon.titan-embed-text-v2:0 ...
+  AccessDeniedException ... not authorized to perform: bedrock:InvokeModel
+```
+
+## Applying them
+
+```bash
+aws iam create-policy \
+  --policy-name BedrockEmbeddingInvoke \
+  --policy-document file://infra/aws/iam/bedrock-embedding-invoke.json
+
+aws iam attach-group-policy \
+  --group-name AI-Lab-Group \
+  --policy-arn arn:aws:iam::<account-id>:policy/BedrockEmbeddingInvoke
+```
+
+Same two commands for the bootstrap policy, then `aws iam
+detach-group-policy` once model access shows as granted.
+
+## What is documented vs. what is confirmed
+
+Confirmed by running the commands above: the current permission state, and
+that both `ListFoundationModels` and `InvokeModel` are denied.
+
+Taken from AWS documentation and **not yet confirmed against this account**:
+
+- that the action list in each policy is sufficient — no live call has
+  succeeded yet, so "sufficient" is unproven either way;
+- that `cohere.embed-v4:0` needs a Marketplace subscription in this account.
+  It is a third-party model, so the bootstrap policy provides for it;
+- the foundation-model ARN form `arn:aws:bedrock:us-east-1::foundation-model/<id>`
+  (no account number) — this is the form AWS's own denial message returned,
+  so it is corroborated;
+- whether the account has an SCP or permissions boundary that would still
+  deny Bedrock after these policies are attached. Nothing here can rule that
+  out from inside the account.
+
+Region is pinned to `us-east-1` to match the configured region. Widening to
+`arn:aws:bedrock:*::foundation-model/...` is a one-line change if the
+benchmark moves region, but it should be a deliberate one.
@@ -0,0 +1,35 @@
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Sid": "ListModelsToConfirmAccess",
+      "Effect": "Allow",
+      "Action": [
+        "bedrock:ListFoundationModels"
+      ],
+      "Resource": "*"
+    },
+    {
+      "Sid": "ReadTheTwoEmbeddingModels",
+      "Effect": "Allow",
+      "Action": [
+        "bedrock:GetFoundationModel"
+      ],
+      "Resource": [
+        "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0",
+        "arn:aws:bedrock:us-east-1::foundation-model/cohere.embed-v4:0"
+      ]
+    },
+    {
+      "Sid": "InvokeOnlyTheTwoEmbeddingModels",
+      "Effect": "Allow",
+      "Action": [
+        "bedrock:InvokeModel"
+      ],
+      "Resource": [
+        "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0",
+        "arn:aws:bedrock:us-east-1::foundation-model/cohere.embed-v4:0"
+      ]
+    }
+  ]
+}
@@ -0,0 +1,41 @@
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Sid": "ReadModelAccessState",
+      "Effect": "Allow",
+      "Action": [
+        "bedrock:GetFoundationModelAvailability",
+        "bedrock:ListFoundationModelAgreementOffers",
+        "bedrock:GetUseCaseForModelAccess",
+        "bedrock:PutUseCaseForModelAccess"
+      ],
+      "Resource": "*"
+    },
+    {
+      "Sid": "AcceptTheModelAgreement",
+      "Effect": "Allow",
+      "Action": [
+        "bedrock:CreateFoundationModelAgreement"
+      ],
+      "Resource": [
+        "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0",
+        "arn:aws:bedrock:us-east-1::foundation-model/cohere.embed-v4:0"
+      ]
+    },
+    {
+      "Sid": "MarketplaceSubscribeOnlyViaBedrock",
+      "Effect": "Allow",
+      "Action": [
+        "aws-marketplace:ViewSubscriptions",
+        "aws-marketplace:Subscribe"
+      ],
+      "Resource": "*",
+      "Condition": {
+        "StringEquals": {
+          "aws:CalledViaLast": "bedrock.amazonaws.com"
+        }
+      }
+    }
+  ]
+}