Evals for a credit policy
Policy tests
21 test applicants, each with the decision the default policy should give and the reason, worked out by hand. The same cases run in CI on every change, so code that shifts a decision fails the build. Edit a rule in the rules console and this page shows who it would affect.
- Matching the default policy
- 21 of 21
- Decided differently
- 0 none
- Approvals
- 11 of 21
Test applicants
Active policy: default.
| Applicant | Expected | Engine | Result |
|---|---|---|---|
| Strong applicantTier 1 STEM master's, established credit, income covers the loanEvery factor at maximum: 25 + 15 + 20 + 20 + 10 + 10 = 100, above the band A floor of 80. | Approved, band A | Approved, band AScore 100 | Match |
| Thin credit fileTier 1 STEM master's, thin credit, $30K income for a $45K loanThin credit scores 12 of 20 and income covers 44% of the target, 8.9 of 20. Total 79.4: approved in band B, just short of A. | Approved, band B | Approved, band BScore 79.4 | Match |
| BorderlineTier 3 school, other field, $15K income for a $60K loan10 + 8.3 + 20 + 3.3 + 6 + 7 = 54.6. Above the review line of 50, below approve at 65, so a credit analyst decides. | Sent to review | Sent to reviewScore 54.6 | Match |
| Low scoreTier 3 MBA, thin credit, $15K income for a $100K loan10 + 12.1 + 12 + 2 + 6 + 7 = 49.1, just under the review line. Borrowing less would lift income coverage. | Declined | DeclinedScore 49.1 | Match |
| IneligibleUK destination, admission not confirmed, unranked schoolFails three knockout rules before any scoring: no UK partner, no confirmed admit, school below tier 3. | Declined, ineligible | Declined, ineligible | Match |
| Entry bandTier 2 business masters, thin credit, $20K income for a $50K loan17.5 + 12.8 + 12 + 5.3 + 8.5 + 10 = 66.1: approved, but only in band C (65 to 69.9). | Approved, band C | Approved, band CScore 66.1 | Match |
| No credit historyStrong profile with no credit file at allNo history scores 6 of 20, but school, field, income and experience carry it: 86, band A. The model underwrites future earnings, not past borrowing. | Approved, band A | Approved, band AScore 86 | Match |
| Medium country riskSame as the thin-file case, citizen of a medium-risk countryCountry risk drops from 10 to 7, so 79.4 becomes 76.4. Still band B. | Approved, band B | Approved, band BScore 76.4 | Match |
| PhD, no household incomeTier 2 medicine PhD, established credit, zero income17.5 + 15 + 20 + 0 + 6 + 10 = 68.5. Approved in band C with no income support at all. | Approved, band C | Approved, band CScore 68.5 | Match |
| Canada destinationStrong applicant studying in CanadaScores 100 like the strong case and routes to the Canada partner, not the US one. | Approved, band A | Approved, band AScore 100 | Match |
| UndergraduateTier 1 STEM undergraduate, graduate-only rule switched offThe graduate-only rule is disabled by default, so it is skipped. Field score falls to 10.5 (0.7 multiplier) and zero experience scores 4: 89.5, band A. | Approved, band A | Approved, band AScore 89.5 | Match |
| Below the loan floorStrong applicant asking for $3,000The floor is $5,000, so the amount rule knocks it out before scoring. | Declined, ineligible | Declined, ineligible | Match |
| Exactly at the floorStrong applicant asking for exactly $5,000The floor is inclusive, so $5,000 passes and the profile scores 100. | Approved, band A | Approved, band AScore 100 | Match |
| Exactly at the ceilingStrong applicant asking for exactly $100,000The ceiling is inclusive. Income now covers 40% of the target, 8 of 20: total 88, band A. | Approved, band A | Approved, band AScore 88 | Match |
| Above the loan ceilingStrong applicant asking for $150,000Above the $100,000 ceiling, so the amount rule declines it. | Declined, ineligible | Declined, ineligible | Match |
| UK destinationStrong applicant studying in the UKNo active UK partner, so the destination rule declines it. | Declined, ineligible | Declined, ineligible | Match |
| Admission not confirmedStrong applicant without a confirmed admitA confirmed admit is a hard requirement. | Declined, ineligible | Declined, ineligible | Match |
| Already graduatedStrong applicant whose program has endedExpected graduation must be in the future. | Declined, ineligible | Declined, ineligible | Match |
| Graduation 5 years outStrong applicant graduating in 60 monthsThe enrollment window is 48 months. Income that far out is too distant to underwrite. | Declined, ineligible | Declined, ineligible | Match |
| Graduation exactly 48 months outStrong applicant at the edge of the enrollment windowThe 48-month window is inclusive, so it passes and scores 100. | Approved, band A | Approved, band AScore 100 | Match |
| Unranked schoolStrong applicant at an unranked schoolThe minimum is tier 3, so an unranked school (tier 4) is declined. | Declined, ineligible | Declined, ineligible | Match |
Properties checked in CI
Beyond the fixed cases, the test suite generates hundreds of random applicants and checks rules that must hold for every one of them. These run in the build, not on this page.
- The same application always gets the same decision, score and trace.
- Scores stay between 0 and 100 and equal the sum of their factors.
- More household income, better credit, a better school or a smaller loan never lowers the score.
- Switching an eligibility rule off never makes an outcome worse.
- An approval always carries a price and a partner; a knockout always names the rule that failed.
- A higher score never gets a higher rate.
- Every rule that is on appears exactly once in the trace.
- The API rejects malformed applications and rule sets with a 400, never a crash.