Who this is for: SaaS buyers, IT admins, procurement leads, and curious product managers who need a fast, defensible answer to “Is this vendor safe enough for our use?” Updated for July 2026 realities — wider adoption of phishing‑resistant authentication, more vendors offering private inference/confidential compute, and growing expectations around software supply‑chain transparency.

What you'll learn: how to build an evidence‑backed SaaS vendor risk scorecard that now includes software‑supply chain (SBOM) checks, AI model provider mapping and model cards, confidential computing options, and concrete mitigations you can apply during pilots.

Why this matters in July 2026: SaaS is no longer just “will they get hacked?” The stakes now include who sees your prompts, where model inference runs, whether an SBOM or vulnerability disclosure exists for the vendor’s stack, and whether you can realistically extract full data and audit logs if the vendor is acquired or shuts down. A scorecard turns opaque marketing into auditable decisions you can defend to procurement, risk, and legal.

Prerequisites and context (what to know before you start)

You don’t need a security PhD, but you should be fluent in a few terms vendors will use. If any are unfamiliar, pause and look them up — they’ll come up in contracts and admin consoles.

  • SSO (Single Sign‑On): login via your identity provider (IdP) such as Microsoft Entra ID, Okta, or Google Workspace.
  • SCIM (System for Cross‑domain Identity Management): automated provisioning/deprovisioning of users and groups from your IdP.
  • MFA (Multi‑Factor Authentication): require at least one additional factor beyond passwords; prefer phishing‑resistant options such as FIDO2 / WebAuthn (widely supported by IdPs and devices in 2026).
  • SOC 2 Type II / ISO 27001: independent audit reports that show controls operate over time. Useful but not sufficient — they rarely speak to plan‑level features or AI prompt handling.
  • DPA (Data Processing Agreement): contract clauses about processing, subprocessors, breach notification timelines, and now increasingly AI‑specific clauses (no‑train, model use, logging).
  • Confidential computing: hardware and software approaches that protect data in memory during processing (Intel SGX, AMD SEV, or cloud‑vendor confidential VMs/enclaves). By 2026 these are offered as production options by major cloud vendors for private inference.
  • SBOM (Software Bill of Materials): a manifest of components the vendor uses. Expect it for higher‑risk tools; it helps you assess supply‑chain vulnerability exposure.
  • Model cards / Datasheets: concise documents that describe an ML model’s purpose, training data risks, limitations, and recommended uses — helpful when vendors rely on third‑party models.

Time box: Target 60–90 minutes for a first pass on mid‑tier purchases ($20–$50/user/month). For regulated data (PHI, payments, student records) run a formal review in parallel — the scorecard is a gating tool, not the final legal sign‑off.

What you’ll need:

  • A spreadsheet (Google Sheets/Excel) or a lightweight system like Notion/Airtable.
  • Access to vendor trust centers, security docs, and the ability to request SOC 2 reports, SBOMs, pen test summaries, and the DPA.
  • Admin access to your IdP recommended to validate SSO/SCIM behavior in practice; a sandbox account with the vendor for export tests.

Step 1: Define “what could go wrong” for this SaaS

Scorecards that try to be universal fail. Treat each product like a vehicle: a password manager is a bank safe; a design app is a collaborative studio. Define the threat model before you start scoring.

  1. Write a one‑line use case. Example: “Customer success team uses Tool X to manage support tickets that include customer PII and screenshots.”
  2. List data types the tool will store/process:
    • Public: marketing assets
    • Internal: roadmaps, non‑public docs
    • Confidential: source code, incident reports
    • Personal data: names, emails, IP addresses
    • Regulated: payment card data, protected health information (PHI), student data
    • Derived/AI prompts: inputs and outputs that might be sent to model providers
  3. Identify access patterns. Employees, contractors, customers, partner access, external integrations (email, cloud storage, CRM), service‑to‑service tokens.
  4. Decide the risk tier:
    • Tier 1 (Low): no personal/confidential data; internal process tooling only.
    • Tier 2 (Medium): personal data or broad internal access.
    • Tier 3 (High): regulated data, security‑sensitive workflows, or deep system integrations (SSO provisioning for many users, API keys to production systems).

Why this matters: Weighing priorities (security vs. reliability vs. privacy) must mirror data sensitivity. Don’t debate uptime for a tool that only processes public assets.

Step 2: Build the scorecard categories (2026 update)

Failure modes have evolved: add explicit supply‑chain checks and stronger AI observability requirements. Here’s a recommended set for Tier 2 tools with default weights you can adjust by tier.

  • Security & Access Controls — 22%
  • Privacy & Data Handling — 18%
  • AI/ML Data Handling & Model Transparency — 18%
  • Software Supply Chain & SBOM — 10%
  • Compliance & Assurance — 10%
  • Reliability & Incident Readiness — 12%
  • Vendor Viability & Exit — 10%

Analogy: Think of buying a house: locks (security), deeds (privacy/contract), skylights (AI) and the foundation (supply chain). In 2026, the foundation often includes open‑source components you don’t see without an SBOM.

Step 3: Choose a scoring model you’ll actually use

Simplicity and reproducibility beat fancy math. The aim is comparability and defensibility.

  1. Use a 0–3 scale per control: 0 = No evidence; 1 = Partial/weak; 2 = Baseline; 3 = Best practice with evidence.
  2. Include an evidence column: link to trust center pages, SOC 2 bridge letters, DPA clauses, SBOM snapshot, model cards, or screenshots of admin settings. Sales slides are not evidence.
  3. Add notes/conditions: capture mitigations like “enable SSO before onboarding” or “limit to non‑PII during pilot.”
  4. Set approval thresholds:
    • Approve: weighted score ≥ 2.3 and no hard stops
    • Approve with conditions: 1.8–2.29 or fixable gaps documented and time‑bound
    • Reject: < 1.8 or a hard stop fails

Hard stops (recommended for Tier 2+):

  • No MFA enforcement and no SSO available at the plan you will buy.
  • No documented export path or export is behind a costly paywall.
  • Vendor refuses to sign a DPA or to disclose subprocessors; no ability to request SBOM for hosted software.
  • AI features that route customer prompts to third‑party models with no contractual “no‑train” option or logging/audit rights.

Step 4: Score Security & Access Controls

This prevents common failure modes: orphaned accounts, leaked tokens, and privilege creep.

  1. SSO support (SAML 2.0 / OIDC): 0: No; 2: Available on your plan; 3: SSO + conditional access guidance + prebuilt IdP templates and SCIM tested in your IdP.
  2. SCIM provisioning: 0: Manual only; 2: Basic SCIM create/deactivate; 3: SCIM + group/role mapping + provisioning audit logs and deprovisioned artifact cleanup.
  3. MFA enforcement: 0: Optional; 2: Admin can enforce MFA; 3: Phishing‑resistant MFA supported and enforceable (passkeys/FIDO2/WebAuthn), per NIST guidance.
  4. RBAC and least privilege: 0: Everyone admin; 2: Standard roles; 3: Custom roles + scoped API tokens + short‑lived credentials and secrets rotation guidance.
  5. Secrets & token handling: 0: No controls; 2: Token revocation and rotation described; 3: Short‑lived tokens, fine‑grained scopes, and recommended vault integrations.
  6. Audit logs and SIEM integration: 0: No logs; 2: Login/admin events logged; 3: Exportable logs + SIEM/EDR integration, structured event schema, and retention controls.

Practical note: If SSO is only available on an enterprise plan costing 3–5× your budget, score the reality you’ll buy. Security features behind a paywall are like seat belts stored in the trunk.

Step 5: Score Privacy & Data Handling

Focus on residency, subprocessors, retention, and contractual commitments. By 2026, many vendors have updated DPAs to include AI specifics — ask for them.

  1. Data residency and regional controls: 0: No info; 2: Documented regions (US/EU/APAC); 3: Customer‑selectable region with clarity on backups, logs, and cross‑region failover behavior.
  2. Subprocessor transparency and change notifications: 0: None; 2: Public list + notifications; 3: Granular mapping of roles and data each subprocessor touches and a contractually required notice period.
  3. DPA availability and AI clauses: 0: Won’t sign; 2: Standard DPA; 3: DPA with breach timelines, subprocessors, and explicit AI/no‑train and prompt‑logging commitments.
  4. Retention and deletion: 0: Vague; 2: Documented deletion timelines; 3: Configurable retention + deletion APIs and verified deletion across backups and logs.
  5. Cross‑border transfer mechanisms: 0: Not disclosed; 2: Standard mechanisms (SCCs, adequacy); 3: Detailed mapping and contractual safeguards for transfers and subprocessors.

Step 6: Score AI/ML Data Handling & Model Transparency

AI is often the trickiest category: vendor assistants can appear helpful while quietly sending prompts elsewhere. In 2026, expect vendors to provide model cards and mapping of which provider handles which feature.

  1. Model provider disclosure and mapping: 0: Vague “third‑party models”; 2: Vendor names model providers for major features; 3: Full mapping (feature → model provider → location of inference) and model cards available.
  2. No‑train / contractual restrictions: 0: No option; 2: Customer can opt out in settings; 3: Contractual “no‑train” plus audit rights and verifiable logs showing opt‑out honored.
  3. Prompt/output retention and auditability: 0: Unknown; 2: Documented retention and admin controls; 3: Configurable retention, deletion APIs, and exportable prompt/output logs for audits.
  4. Private inference / confidential compute: 0: Not supported; 2: Described options; 3: Private inference or confidential compute options (cloud enclave/private instance) with networking isolation.
  5. Redaction and data minimization: 0: None; 2: Basic redaction tools; 3: Built‑in PII redaction, prompt filtering, and least‑privilege agents for model access.

Why this matters: An assistant that routes support tickets to an external model provider without contractual or technical controls creates privacy, IP, and compliance risk. Ask for the model cards and a data flow diagram during procurement.

Step 7: Score Software Supply Chain & SBOM

Supply‑chain risk is now central. An SBOM helps you understand transitive dependencies and exposure to CVEs. Ask for a trending score and update cadence.

  1. SBOM availability: 0: None; 2: General list of OSS components; 3: Machine‑readable SBOM on demand and process for SBOM updates.
  2. Vulnerability management: 0: No policy; 2: Regular scans and patching cadence; 3: Automated vulnerability alerts, CVE mapping, and documented remediation SLAs.
  3. VDP & pen tests: 0: No program; 2: Annual third‑party pen test; 3: Continuous scanning, public vulnerability disclosure program (VDP), and bug bounty or clear triage SLAs.

Tip: If a vendor refuses to provide an SBOM, consider it a risk increase — hidden dependencies are exactly what turns a small breach into a widespread problem.

Step 8: Score Compliance & Assurance

Compliance artefacts are evidence, not guarantees. Use them to corroborate controls you can test.

  1. SOC 2 Type II / ISO 27001: 0: None; 1: SOC 2 Type I; 2: Recent SOC 2 Type II; 3: SOC 2 Type II + ISO 27001 or attestations with clean findings and remediations.
  2. Penetration testing & VDP: 0: No mention; 2: Annual pen test + exec summary; 3: Regular testing, public VDP, and SLAed response.
  3. Questionnaire maturity: 0: No responses; 2: Completed SIG/CAIQ; 3: Questionnaire + on‑demand assessor access or shared evidence via secure portal.

Step 9: Score Reliability & Incident Readiness

Outages and poor incident communication are operational risks. Prioritize transparency and tested processes.

  1. Status page and incident history: 0: None; 2: Public status page; 3: Postmortems with RCA and corrective action plans.
  2. Backups, RTO/RPO and tested DR: 0: Not described; 2: Documented backups; 3: Tested restores, published RTO/RPO, and recovery playbooks.
  3. Incident notification and escalation: 0: No policy; 2: Breach notification in DPA; 3: Dedicated security contact, incident SLAs, and customer notification playbook.

Step 10: Score Vendor Viability & Exit

Lock‑in is often more damaging than a one‑off vulnerability. Verify export fidelity, API maturity, and contractual migration support.

  1. Data export fidelity: 0: No export; 2: CSV/JSON export or API; 3: Full‑fidelity export including attachments, metadata, audit trails, and programmatic export tools.
  2. Integration ecosystem and sandbox: 0: Closed; 2: Webhooks + common integrations; 3: Mature API, SDKs, sandbox, and pagination/rate‑limit clarity for migrations.
  3. Contract clarity and termination support: 0: Unclear; 2: Clear terms and admin controls; 3: Notice periods, migration support, and termination playbook with support hours.

Real example: In vendor evaluations since 2024, teams that ran a pilot export discovered missing comment threads and audit metadata — that’s the sort of fidelity gap that breaks migrations. Run an export test during your trial.

Step 11: Turn the score into a decision and mitigations

After scoring:

  1. Compute the weighted score. (Category average × weight, sum across categories.)
  2. List conditions to approve. Examples for July 2026:
    • Enable SSO + SCIM before onboarding; require quarterly access reviews.
    • Require phishing‑resistant MFA for admin and privileged accounts (FIDO2/passkeys).
    • Contractual “no‑train” or opt‑out for AI features; require prompt logs and a data flow diagram for model inference.
    • Require SBOM on demand and a vendor commitment to remediate critical CVEs within a contractual SLA.
    • Perform an export fidelity test during any pilot and retain a copy of the exported data for verification.
  3. Assign an owner and review cadence. Tier 2: annually or at renewal; Tier 3: every 6 months or after major changes (new subprocessors, acquisition, AI updates, or CVE affecting core deps).

Deliverable to share: one‑page executive summary: purpose, tier, final score, hard stops, conditions, evidence links, owner, and next review date. Keep it short — procurement loves brevity.

Common mistakes (and how to avoid them)

  • Treating SOC 2 as a ticket to “approved.” SOC 2 is evidence, not proof you can enable SSO or disable AI sharing on the plan you’ll buy.
  • Scoring the vendor’s enterprise plan, not the plan you’ll buy. Score the reality.
  • Ignoring supply‑chain risk. Without an SBOM or a VDP, you may be exposed to transitive vulnerabilities.
  • Assuming AI features are private by default. Explicitly ask what goes to model providers, where inference runs, and how long prompts are retained.
  • No owner, no follow up. Without a named owner and a re‑score date, the scorecard rots and decisions become guesswork.

Pro tips (practical, time‑saving, and effective)

  • Keep a “known good” baseline. For Tier 2+: require SSO, SCIM, export, no‑train option, and an SBOM on request.
  • Score “3” only with evidence. Reports, exports, SBOM snapshots, or DPA clauses — not sales slides.
  • Use a short SIG‑lite questionnaire first. A 12–15 question form (SSO/SCIM, encryption, retention, subprocessors, incident contacts, AI handling, SBOM availability) cuts back‑and‑forth.
  • Build compensating controls playbooks. If SCIM is missing, require quarterly manual reconciliations and shorter admin sessions; if prompt logging is absent, restrict tool to non‑sensitive projects during pilot.
  • Do an extraction test during any pilot. Export a representative dataset and verify fidelity (comments, timestamps, attachments, audit trails) before full rollout.

FAQ

Do I still need a scorecard if the vendor publishes a SOC 2 Type II?

Yes. SOC 2 is a useful assurance artifact but rarely covers plan‑level features (SSO/SCIM), AI prompt handling, or supply‑chain transparency. The scorecard translates audit reports into operational de