MamNajem Product Outcomes: implementation and review brief¶
Updated: 2026-10-06. Status: integration and production reporting deployed, publication enabled, and all 524 snapshot rows exactly reconciled in each environment.
The backend owns the versioned product definitions and durable reporting evidence. Admin and the private New Relic MamNajem Product Outcomes dashboard consume the same aggregate results. The owner authorized the existing User key for the Free-plan runtime; no additional token or paid upgrade is required. Both runtimes verify exact publication independently of HTTP ingestion success. Authenticated New Relic visual inspection remains unverified because the available browser sessions redirect to sign-in.
Application context¶
MamNajem is a Polish rental-management application for individual landlords and their tenants. It connects properties, leases, rent schedules, payment evidence, documents, invitations, messages, reminders, handovers, deposits, final settlement, and tenancy history. Its initial audience is self-managing Polish landlords with approximately 2–10 long-term residential units; the preserved onboarding qualification rule accepts portfolio bands from one through 11–25 units. Pilot recruitment and qualification therefore have different boundaries.
Owners are the initial paying customers. Tenants participate in assigned tenancy workflows. Agency teams, enterprise property managers, accountants, and short-term rental operators are outside the ordinary first-release promise. The documented commercial direction includes genuine trial access, fixed plans, pilot extensions, and read-only Archive access. An expressed willingness to pay is feedback; a proven paid term is a separate result.
| Surface | Purpose |
|---|---|
| Public web | Product explanation, pricing direction, contact, pilot intake, and agency waitlist. |
| Mobile owner app | Profile qualification, property and lease setup, rent schedules, payment evidence, documents, handovers, and settlement. |
| Mobile tenant app | Accepted lease access and permitted tasks, messages, receipts, documents, and tenancy confirmation. |
| Backend | Authentication and authorization, business transactions, database history, provider webhook handling, product evidence, and reporting. |
| Admin | Individual account and pilot investigation, explicit classification, recorded pilot assessment, aggregate reports, and audited actions. |
| Architecture | Runtime delivery, private query-secret references, dashboard definitions, and isolated reporting warnings. |
Rental work has a monthly cadence. A landlord may create one lease and return around rent dates or occasional tasks. App opens, automatic reminders, repeated failures, and setup record counts do not prove continued owner value. The recurring-work contracts below count completed approved owner work and show the observation window.
For example, creating a connected property, ACTIVE lease, and usable rent schedule can activate an owner without tenant acceptance. An owner later creates a receipt or changes an obligation between paid and unpaid. That successful transaction can set one owner-month activity flag. A tenant upload, overnight schedule job, admin correction, or no-op retry does not make that owner active. Two qualifying actions still count as one owner in the month.
Selected populations and time rules¶
- Default population: explicitly classified REAL production owners. Integration is available for verification. INTERNAL, TEST, and UNKNOWN populations stay separate.
- Existing accounts and intake records default to UNKNOWN. Account matching does not classify an account. Matched pilot participants inherit account classification for reporting; unmatched participants use their explicit intake classification.
- A known first qualification is captured atomically with the successful backend profile/role transaction and uses qualification version icp-v1. Repeated edits and retries cannot move it. Legacy client or database observations are FIRST_OBSERVED, not proof of first-ever acquisition.
- Q7 is the selected cohort's owners with a known first qualification whose complete seven elapsed days have elapsed at cutoff. Pending owners remain visible. The milestone deadline is inclusive; seven elapsed days are independent of calendar-day or DST boundaries.
- Q7 readiness uses qualification-window evidence. Setup removed before qualification cannot count through a preserved earlier milestone; valid property/connected setup present at qualification counts at qualification. The first global activation remains the recurring-work cohort anchor and is preserved independently.
- P is the deduplicated explicitly real accepted pilot population. Paused and exited participants remain members. Unmatched records remain visible.
- Weeks begin Monday and months use Europe/Warsaw calendar boundaries. Business periods use an inclusive start and exclusive end. Canceled, archived, or inactive owners remain in historical cohorts while their accounts and evidence exist; normal account deletion rules still apply.
- Percentages include their counts. A zero denominator returns
available=falseandvalue=null, displayed as Not available. Pending, unknown, excluded, and unmeasurable counts are separate observations.
The twelve metric contracts¶
| Contract | Shared backend definition | Interpretation and uncertainty |
|---|---|---|
| 1. New qualified owners | Distinct REAL owners with a known first qualification in the selected Warsaw week. | First observed and unknown anchors are separate. This is acquisition, not profile-edit activity. |
| 2. Property readiness | Q7 owners with a real non-archived property at qualification or ready by their deadline / Q7. | A property removed before qualification cannot count from an earlier milestone. Pending qualification windows remain visible. |
| 3. Lease readiness | Q7 owners with an ACTIVE lease connected to that property by their deadline / Q7. | An unrelated property and lease cannot satisfy the chain. Upcoming PENDING leases remain visible and do not remove owners from Q7. |
| 4. Setup activation | Q7 owners with that connected property, ACTIVE lease, and usable active positive-value recurring RENT schedule at qualification or achieved by their deadline / Q7. | Existing valid setup counts at qualification; removed pre-qualification setup does not. The first global activation is preserved separately. Schedule rules and payment-method readiness are respected. OWNER, ASSISTED, IMPORTED, and UNKNOWN provenance stay separate. Tenant acceptance and documents are optional here. |
| 5. Proven paid-term owners | A current positive-value verified unreversed paid term plus effective entitlement. | Disabling renewal does not remove remaining paid access. Trials, complimentary access, test purchases, unproven grace, pilot extensions, and unknown evidence are separate. |
| 6. Trial owners | Genuine effective trial access, excluding owners already counted as paid. | Effective native/provider trials and no-card trials of up to thirty elapsed days count. Longer no-card trials require explicit operator evidence; unsupported provenance stays UNKNOWN. Explicit pilot extensions have their own category. |
| 7. Rental footprint | Distinct current real non-archived properties with ACTIVE leases, and distinct ACTIVE leases. | These are stock counts, not completed work or retention. |
| 8. Pilot statuses | Current operator-assessed status distribution across P, including missing status. | Status is assessment, not an inferred activity flag. |
| 9. Pilot evidence readiness | P with matched application records, connected usable setup, and relevant document/asset evidence / P. | This deliberately requires more evidence than general activation. Unmatched participants remain in P and are shown separately. |
| 10. Pricing responses | Accepted / answered and answered / P, grouped by the recorded plan and price offer. | Missing offer context stays visible. A response is not payment evidence. |
| 11. Recorded support effort | Existing cumulative recorded support hours / P, total hours, and recording coverage. | Cumulative is explicit. Missing hours differ from recorded zero; this does not estimate weekly support burden. |
| 12. Measurement health | Cutoff, publication status, last verified publication, expected/queryable snapshot comparison, reconciliation, classification coverage, and historical-anchor coverage. | A current invalid publication displays unavailable data instead of revealing an older healthy run. |
The calculator also publishes supporting rows for first-observed owners, unknown anchors, pending leases, access categories, activation provenance, pilot participants/unmatched records, and retention members without measurable anchors. These rows explain the twelve contracts rather than adding unrelated product KPIs.
Recurring work and return¶
Monthly observed owner work counts distinct activated REAL owners completing at least one approved owner action during the Warsaw month. Activation before the month, known activation during it, and unknown timing are separate. Setup first observed during a month cannot prove first activation in that month; an earlier observation proves setup already existed before the work month. Approved work is successful owner receipt creation, an actual paid/unpaid obligation transition, owner handover confirmation, or owner settlement confirmation. Trusted backend authentication/service audit context supplies actor provenance. Automatic jobs, admin actions, tenant actions, setup alone, failed or rolled-back writes, retries, and no-op changes do not qualify.
Next-full-month owner return is approved work during the complete month following activation month / the corresponding fully observed activation cohort. If activation occurs in October, the return month is November; the rate becomes available only after November is completely observed. Pending windows and members without known activation anchors remain visible. A cohort predating the dependable collection start is not retroactively treated as fully observed.
Recurring-work and return cohorts use the preserved first global activation, rather than the later qualification-window setup timestamps used for Q7. For example, an owner first activated in September, removed that setup, and qualified in October still belongs to September's activation cohort. October Q7 requires valid setup present at qualification or completed by that qualification's deadline. Recording it does not move the owner's first activation month.
The database retains the current and twelve preceding Warsaw owner-month flags. Aggregate reporting runs persist independently of New Relic. Unsupported earlier activity or first-ever milestones remain unavailable; this change includes no destructive backfill, database reset, churn/MRR calculation, or new analytics infrastructure.
Evidence and payment limits¶
Qualification and setup milestones live in product_owner_evidence. Completed-work flags live in product_owner_month_activity. Their capture shares the relevant business transaction so rollback cannot create successful product evidence. Qualification eligibility still requires OWNER role, an eligible unit band, SELF_MANAGED or MIXED management, and a nonempty city.
Legacy observations without dependable first-ever tracking remain FIRST_OBSERVED. Later profile edits or setup scans do not manufacture first qualification, Q7 setup timestamps, or known retention anchors. Connected bank-transfer setup requires the schedule's selected account or lease payee account to be live and owned by the property's owner; stored IBAN evidence can be encrypted. Account-selection changes participate in transactional capture. General activation, pilot aggregate readiness, and individual pilot evidence share the connected lease rules. Pilot evidence additionally needs a LEASE_DOC on the same connected ACTIVE lease, rather than a document on an unrelated lease.
Payment proof is normalized from existing successfully processed signature-verified Stripe subscription webhook records. Supported invoice.paid evidence requires one complete positive non-prorated recurring invoice line linked to the current subscription, with a known term and verified payment artifact. Legacy in-band charge/PaymentIntent evidence and a supported modern paid PaymentIntent representation preserve amount, currency, live/test environment, and available invoice/charge/payment identifiers inside the application database. Manual, prorated, one-off, truncated, multiple-line, or incomplete evidence remains unsupported. Refunds and disputes preserve reversal evidence, including reversal before a delayed invoice. No provider lookups are added. Store payment evidence without an already-supported amount and term is UNKNOWN, not a guessed paid conversion.
Account-linked evidence follows the existing cascade-deletion relationship. The retained aggregate runs contain no customer identifiers and are independent of telemetry retention. Aggregate history describes the population observed at each run cutoff; it is not a replacement for individual customer investigation.
Shared reports and admin behavior¶
ProductReportCalculator.VERSION is product-outcomes-v2. Version 2 adds the explicit unknown activation-timing group for monthly observed work. Persisted version 1 results retain their original version and payload; deploying a later definition cannot relabel them. ProductReportSource reads bounded source rows sequentially from a single read-only REPEATABLE_READ transaction with a common cutoff, row bound, and time bound. Exceeding a bound fails the run rather than silently sampling customers.
Immutable product_report_runs hold revision, environment, definition version, cutoff, generated time, aggregate metrics, and health. product_report_publications separately hold the exact snapshot payload and mutable delivery state. A database-backed singleton job claim coalesces overlapping refresh requests. A superseded publication cannot replace the current run.
| Admin contract | Result |
|---|---|
GET /v1/admin/product-reports/latest |
Current aggregate report, version, cutoff, counts, maturity/coverage, and publication state. HTTP 204 means no run exists. |
POST /v1/admin/product-reports/refresh |
Audited asynchronous request; HTTP 202 and {runId}. Acceptance does not establish computation or New Relic verification. |
PATCH /v1/admin/accounts/{id}/classification |
Explicit classification and operator reason, with optional owner setup provenance and trial-access evidence. |
| Existing pilot review patch | Explicit intakeClassification; matched account classification remains authoritative for reports. |
Admin's Product Outcomes view replaces the previous locally calculated qualified-owner funnel. It displays shared results with counts, observation uncertainty, publication context, a business-period selector, and Definitions & data quality. Recompute shows the accepted run ID and waits for the current reporting result. Existing individual account and pilot investigation remains in admin.
The owner account editor's Trial access evidence field records UNKNOWN, GENUINE, or PILOT_EXTENSION through the same audited account action. Existing records default to UNKNOWN. A no-card trial longer than thirty elapsed days requires explicit provenance before it can count as genuine or as a pilot extension; a long expiry or participant match is insufficient. The assessment classifies existing effective access and preserves its entitlement and dates. Provider paid terms still require the supported verified payment proof described above.
Publication and the private dashboard¶
Collection/report calculation are enabled independently of publication. The reporting schedule is daily at 03:00 Europe/Warsaw, with an initial run when enabled. The prepared private dashboard has Overview and Definitions and data quality pages, defaults to production, and includes integration verification. No customer-specific investigation is exported.
The export constructs only MamNajemProductReportSnapshot aggregate rows and MamNajemProductReportRun status markers. Each includes the run identity, revision, definition/cutoff context, canonical environment, and explicit business periods. Controlled dimensions cover provenance, activation timing, observation/maturity, pilot status, and offer context. No account identifiers, hashes, emails, contact details, addresses, or free text are allowed.
Daily republication includes the supported 13-month business window with current event timestamps. Historical periodStart/periodEnd remain attributes. New Relic requires event timestamps within 24 hours of server time and drops null attributes, so unavailable values use the explicit availability flag. The dashboard uses cohort/month tables and ignores the global time picker for snapshot widgets. See the custom-event requirements.
The Event API is asynchronous. HTTP acceptance does not prove delivery. The backend polls every 30 seconds for up to ten minutes and compares every expected logical snapshot key and allowlisted value against the persisted result. Identical duplicate delivery retries collapse; conflicting values, missing or duplicate logical keys, and reconciliation differences fail verification. Failed publication retries every fifteen minutes only while the run is current; its snapshot payload and timestamp stay unchanged. After 24 hours an old event timestamp cannot be resent as a valid current event: invalidate that publication and await or request a fresh run. See the Event API contract.
Snapshot rows carry publicationKey=runId|VERIFIED. Authoritative current status markers carry runId|status. Every widget selects the newest current marker before availability/status filtering. PENDING, FAILED, INVALID, and DISABLED markers therefore cannot silently expose older healthy snapshot data. Verification includes current-marker visibility as well as row reconciliation.
Latest backend responses and publication-status retries preserve each immutable result's stored definition version and environment. Deploying new application definitions cannot relabel a previous result.
Failed computation markers have a dedicated database-backed retry even when no result or publication row exists. The minute monitor claims the current FAILED run after fifteen minutes under a two-minute lease and retries the same persisted payload/timestamp with a one-minute delivery bound. Current-run guards and cleared retry state on a new reservation prevent delayed obsolete delivery.
Actual operational boundary¶
On 2026-10-06 the owner confirmed the New Relic Free plan and explicitly authorized using the existing administrator User key for backend verification. This replaces the original separate read-only identity requirement for the current deployment. The key retains administrator permissions; runtime queries are fixed to account 8573192 EU, and private delivery uses the two existing environment-specific AWS query-secret containers separately from the ingest license key. Never print, commit, place in Terraform state, or include credential values in evidence.
Free includes 100 GB/month of data ingestion. The read-only account inspection measured 7.799118108 GB over the preceding 30 days and effective custom-event retention of 30 days, with no overrides. This allowance is not a paid upgrade or a promise that future usage cannot grow; monitor the existing account usage. See New Relic pricing. Daily republication keeps the supported thirteen business months queryable through explicit historical attributes; the backend independently retains aggregate runs.
Current unrestricted backend mvn verify passes 6,790 tests with zero failures, errors or skips, coverage and SpotBugs (zero findings). Architecture build and all 41 suites / 451 tests pass. Admin passes 232 tests (149 UI, 49 Node, 34 Python) and build. All 162 backend CI-script Python tests pass. Docs pass 98 tests in a canonical CI checkout, strict build and artifact scan. The original workspace is 97/98 solely because the preserved existing local migration branch violates the scanner's active-ref namespace policy; the branch and scanner remain unchanged.
Publication was enabled on 2026-10-06 after the owner confirmed the Free plan and explicitly authorized reusing the existing administrator User API key. Both managed runtime query-secret containers now have an AWSCURRENT version. The existing backend CI metadata credential is protected, masked and hidden in the exact production environment; the metadata check remains enabled. No new New Relic user, custom group, paid upgrade or paid secret-store service was created.
The first enabled acceptance runs use product-outcomes-v2, with 524 expected and 524 queryable rows, exact reconciliation and no reporting failure state:
| Environment | Run | Revision | Cutoff (UTC) |
|---|---|---|---|
| Integration | ba8a9b82-daaf-496a-a316-e1fa6eb16f19 |
8 | 2026-10-06T07:49:28.616486Z |
| Production | d92c6108-143d-4c40-99d3-1cd655584424 |
4 | 2026-10-06T07:56:19.202590Z |
These immutable acceptance runs were produced by backend 29e6b74ad8ff0903ce2c731a8014024299ae667e. Integration's exact deployment retry succeeded through job 16960487050 in pipeline 2915601096. Production's exact deployment retry succeeded through job 16960582118 in pipeline 2915666448, followed by successful deployment verification pipeline 2916691643. ECS task definition 20 completed its rollout with one desired/running task, no pending task, publication enabled and the private query-secret reference installed. The checked production startup stream contains no application ERROR or reporting WARN. Later daily or authorized refresh runs have new run IDs; inspect the current authoritative marker rather than treating this evidence table as the current run selector.
Authenticated integration admin results match the persisted report: 13 history months, 13 unknown-activation-timing rows, and 372 UNKNOWN owners. An independent comparison matched all 524 exact snapshot keys and 13,624 allowlisted attribute values against the backend API result. The runtime's automatic exact key/value comparison independently verified both environments. Both enabled heartbeats and zero reporting-failure ages are queryable. All eleven live dashboard NRQL widgets in each environment return results and use the current VERIFIED run; the private dashboard remains PRIVATE and its query widgets ignore the global time picker.
The available New Relic browser profiles redirect to sign-in. Configuration and all 22 live widget queries are verified through NerdGraph, but visual inspection inside the authenticated New Relic UI remains unverified. This does not require another API key. Authenticated admin report/definitions rendering was verified independently. Unsupported historical anchors remain unavailable, and existing UNKNOWN accounts require explicit operator classification before contributing to REAL adoption metrics.
Initial deterministic TEST fixture acceptance verified qualification retries, connected setup, owner receipt work, retry idempotency, tenant denial and exclusion from REAL metrics. Supported application deletion and Cognito cleanup removed the fixtures. The admin integration overview and definitions rendered the shared versioned results without browser errors; admin production deploy pipeline 2915476310 and its exact build marker were also verified. No production QA customer was created.
Source map¶
Paths below are relative to the workspace repository named in the first column.
| Repository | Actual source | Responsibility |
|---|---|---|
| Backend | src/main/resources/db/changelog/0016-product-evidence.sql |
Classification, qualification-window setup evidence, preserved first global activation, owner-month flags, existing payment normalization, and account-linked evidence. |
| Backend | src/main/resources/db/changelog/0017-product-report-runs.sql |
Immutable results, separate publication state/payloads, revision sequence, singleton/current-run claim state. |
| Backend | src/main/java/pl/mamnajem/service/ProductReportCalculator.java |
One shared versioned contract and Warsaw/maturity calculations. |
| Backend | src/main/java/pl/mamnajem/service/ProductReportSource.java |
Bounded consistent database snapshot and source reconciliation inputs. |
| Backend | src/main/java/pl/mamnajem/service/ProductReportExport.java |
Aggregate allowlisting, logical keys, and exact value comparison. |
| Backend | src/main/java/pl/mamnajem/service/ProductReportNewRelicClient.java |
EU Event API and separate account-scoped query verification. |
| Backend | src/main/java/pl/mamnajem/service/ProductReportStore.java, ProductReportService.java; src/main/java/pl/mamnajem/controller/AdminProductReportsController.java |
Database claims/current-run state, scheduled calculation/publication, and audited admin APIs. |
| Backend | src/main/java/pl/mamnajem/config/ProductReportProperties.java |
Collection/publication defaults, environment/account/region, private runtime inputs, row bound. |
| Backend | .gitlab/ci/deploy-integration-host.sh |
Protected integration query-secret ARN, disabled publication default, and private exact-ARN AWSCURRENT handoff on the host. |
| Backend | .gitlab/ci/deploy-production-ecs.sh, .gitlab/ci/verify-production-product-report-variable-metadata.sh, .gitlab/ci/product-report-runtime.md |
Protected exact-environment production inputs, disabled default, and private ECS query-secret reference handoff. |
| Backend | src/main/java/pl/mamnajem/dto/ProductReportMetric.java, ProductReportResponse.java |
Shared aggregate response shape. |
| Admin | src/ProductReports.tsx, src/App.tsx, src/lib/api.ts |
Aggregate report UI, classifications, accepted refresh, and investigation APIs. |
| Architecture | scripts/configure-newrelic-product-report.cjs |
Private dashboard, authoritative current selectors, and non-paging warnings. |
| Architecture | terraform/aws-integration/product-report.tf; terraform/aws-production/product-report.tf, storage.tf, runtime.tf |
Dedicated private query-secret reference and disabled manual publication wiring. |
| Architecture | docs/runbooks/newrelic-product-report.md |
Architecture provisioning and account inspection procedure. |
Review and acceptance¶
Review denominator stability, historical uncertainty, connected setup, proven payment, pilot deduplication/offers, cumulative support coverage, privacy allowlisting, and invalid-current-run behavior. The defaults are selected; do not treat the earlier broad candidate metrics as required scope.
Focused checks cover qualification retries/edits, legacy anchors, connected/existing setup, exact seven-day deadlines, owner/admin/system/tenant provenance, rollback/no-ops, Warsaw/DST/maturity, canceled or archived cohorts, payment evidence/reversal/grace, pilot deduplication/offers/support coverage, concurrent claims, delayed ingestion, conflicting/missing keys, supersession, authorization, and export fields. Delivery gates remain backend mvn verify, admin checks/build, architecture build/tests, and docs checks/build.
Integration acceptance must compare deterministic TEST fixtures, backend aggregate results, queryable New Relic rows, and rendered dashboard behavior without including fixtures in the REAL population. Record pushed pipeline terminal states and the verified published documentation page. The authorized manual production rollout and exact New Relic row comparison are complete as recorded above. Authenticated visual inspection inside the private New Relic UI remains unverified; all configured queries are verified directly.