| Document item | Detail |
|---|---|
| Student | ERIC KIMATHI |
| Project | SurveyFlow |
| Domain | Social innovation and organizational research |
| Repository | https://github.com/Ericokim/SurveyFlow |
| Live application | https://surveyflow-eric.netlify.app |
| API & interactive reference | https://surveyflow-api.onrender.com |
| API health check | https://surveyflow-api.onrender.com/api/health |
| Demo walkthrough | Watch the video |
| Report date | [date] |
- SurveyFlow — Cloud-Based Survey & Feedback Management System
- Table of Contents
- Executive Summary
- 1. Problem Identification & Research
- 2. Product Definition
- 3. Requirements & User Workflows
- 4. System Architecture
- 5. Design & Implementation
- 6. Security, Privacy & Accessibility
- 7. Testing, CI & Quality
- 8. Cloud Deployment & Operations
- 9. Evaluation, Risks & Roadmap
- 10. Ten-Minute Presentation Plan
- 11. Local Setup & Maintenance
- References
- Appendix A: Checkpoint Evidence Map
SurveyFlow is a survey management system for organizations that need a configurable platform for creating, publishing, distributing, and analyzing surveys. It supports authenticated administrators, public respondent links, custom branding, recipient whitelisting, SMS invitations, dynamic survey logic, analytics, and exports, so a team can run a branded survey end to end without hard-coding a form for every questionnaire. Its builder is schema-driven: the same editor and renderer produce either a single-question flow or a grouped, section-based survey from one stored definition, so the layout is a configuration choice rather than a separate build.
The system is a full-stack MERN application — a React 19 / Vite client, a Node.js / Express 5 API, and MongoDB with Mongoose — with the client deployed on Netlify, the API on Render, and production data on MongoDB Atlas. The live application is publicly reachable, the production build succeeds, and 117 unit tests pass across the client and server. Documented limitations (no CI pipeline, an open CORS policy, browser-stored JWTs, and role authorization that is defined but not yet enforced) are set out honestly in §6 and §9 as the next hardening priorities.
Organizations across education, health, community programmes, and research depend on structured feedback to make decisions, but collecting it well is harder than it looks. They have to design valid questions, reach the right audience, keep out duplicate or ineligible responses, adapt later questions to earlier answers, preserve exactly what each respondent saw, and turn raw submissions into something they can act on. In practice this work is scattered across spreadsheets, messaging apps, and one-off forms, which duplicates effort and makes results hard to trust.
The popular tools each solve part of this and stop. Google Forms and Microsoft Forms are free and fast, but they offer only shallow branching, no real notion of an eligible-recipient list, weak branding, and no version integrity — editing a live form silently changes what later respondents see. SurveyMonkey and Typeform add nicer question flows and presentation, but advanced logic, respondent control, and exports sit behind paid tiers, and analysis still lives apart from distribution. Qualtrics is powerful but heavy and expensive — overkill for a small team that just needs a credible, branded survey with controlled access. Building a bespoke form per study closes the fit gap but is slow and costly to change.
SurveyFlow targets the middle ground those tools leave open: a single configurable platform where a survey is stored as data, not code. Sections, questions, validation, and branching rules are persisted and rendered dynamically, so one system serves many questionnaires, adding a capability is a configuration change rather than a rebuild, and eligibility, versioning, branding, distribution, and analytics are first-class rather than add-ons.
- Feedback data has public value when collected consistently and acted on. The World Bank frames survey and citizen-generated data as a public good that supports evidence-based improvement. [verify — World Bank, WDR 2021 / Service Delivery Indicators] → For SurveyFlow: keeping collection and analysis in one workflow helps feedback lead to action rather than sitting in disconnected files.
- No single survey layout is optimal for every study. Research on mobile web questionnaires reports that item grouping and scrolling affect break-off, and page size is not one-size-fits-all. [verify — Mavletova & Couper, 2016] → For SurveyFlow: both single-question and grouped, section-based presentation are supported instead of forcing one format.
- Response format affects data quality. A study of online response formats found simpler, carefully constructed formats can outperform others on data-quality measures. [verify — JSSAM response-format study] → For SurveyFlow: the schema-driven renderer plus server-side validation keep answers clean rather than trusting the client display.
- reusable authoring instead of rebuilding each instrument;
- configurable presentation plus conditional logic;
- controlled distribution and response integrity;
- analysis close to the collection workflow;
- privacy and accessibility built in from the start.
Enable small and medium organizations to run credible, branded, logic-driven surveys without assembling separate form, recipient, messaging, and reporting systems.
| User | Job to be done | How SurveyFlow responds |
|---|---|---|
| Research administrator | Create and preserve an instrument without code | Schema-driven editor, validation, preview, versioned publishing |
| Operations officer | Reach a known audience and track participation | CSV upload, whitelist, SMS invitations, recipient status |
| Analyst | Understand completion and answer patterns | Dashboard, analytics, filters, exports |
| Respondent | Complete only relevant questions | Public link, conditional flow, validation, save/resume |
The code defines admin and viewer roles, but the routed product currently distinguishes
authenticated administrators from public respondents; granular per-role authorization is future work
(see §6).
Implemented: authentication and password recovery, company/branding settings, the full survey lifecycle (draft → publish → close), soft delete and restore, seven question types, sections, preview/test modes, conditional logic, versioned publishing, recipient upload and whitelisting, progress save, one-response controls, SMS distribution, analytics, and exports.
Out of scope for this release: billing, enterprise SSO, real-time collaborative editing, a native mobile app, offline capture, multilingual authoring, advanced statistical modelling, and a complete role-permission administration UI.
| Measure | Initial target | Why it matters |
|---|---|---|
| First survey published without support | ≥ 85% | Authoring clarity |
| Median simple-survey setup time | < 10 min | Efficiency |
| Valid final-submission rate | ≥ 95% | Respondent UX |
| Public response availability | 99.5% monthly | Reliability |
| Critical accessibility/security defects at release | 0 | Inclusive, safe use |
| ID | Requirement | Evidence | Status |
|---|---|---|---|
| FR-01 | A user can create a workspace and authenticate | Auth pages, JWT API, company creation | Implemented |
| FR-02 | An administrator can create and edit a survey without code | Schema-driven editor and survey API | Implemented |
| FR-03 | A survey supports sections, common question types, validation, and logic | Seven renderers, embedded schemas, twin logic engines | Implemented |
| FR-04 | An administrator can preview before publishing | Draft, test, and published-preview routes | Implemented |
| FR-05 | Publishing preserves the questionnaire shown to respondents | SurveyVersion snapshots + Response.surveyVersion |
Implemented |
| FR-06 | Access can be open or restricted to recipients | Whitelist gate, hashed identifier lookup, blacklist | Implemented |
| FR-07 | Respondents can save progress and submit valid answers | Progress and submission endpoints | Implemented |
| FR-08 | An organization can distribute and track invitations | Recipient management, SMS service, communication logs | Implemented |
| FR-09 | Administrators can review and export outcomes | Analytics, response management, exports | Implemented |
| FR-10 | The system exposes a documented API | OpenAPI 3.1 + interactive Scalar reference | Implemented |
| FR-11 | Every change is automatically tested before deployment | CI workflow | Not implemented |
Administrator: authenticate → create → add sections/questions → configure logic → preview → optionally upload recipients → publish/share → monitor → analyse/export → close.
Respondent: open public link → pass the whitelist gate when enabled → answer visible questions → follow conditional navigation → save/resume when identified → submit → see the configured thank-you.
Lifecycle: draft → preview/test → publish (immutable version) → live collection → close. Surveys can also be duplicated, soft-deleted, or restored; test and live submissions stay separate.
SurveyFlow is a two-tier MERN application: a React 19 / Vite single-page client and an Express 5 REST API over MongoDB, with AWS S3 for asset storage and Africa's Talking for SMS.
flowchart TB
Admin[Administrator browser] --> Client[React 19 + Vite SPA]
Respondent[Respondent browser] --> Client
Client --> Router[TanStack Router] --> Query[TanStack Query + Axios] --> API[Express 5 REST API]
API --> Guard[Helmet · CORS · request ID · auth · validation · errors] --> Domain[Controllers & services] --> Models[Mongoose models] --> Mongo[(MongoDB Atlas)]
Domain --> S3[AWS S3 assets]
Domain --> SMS[Africa's Talking SMS]
API request pipeline (server/server.js), in order: trust proxy → Helmet → compression →
Morgan request logging (streamed to Winston) → request-ID middleware → CORS → JSON/urlencoded body
parsing (10 MB limit) → routes mounted at /api → notFound → central error handler. The interactive
API reference (Scalar) and /openapi.json are served at the root under a relaxed CSP; all feature
routes mount from server/routes/index.js.
Client provider tree (client/src/app/providers.jsx), outer to inner: theme → alert → TanStack
Query client → tooltip → TanStack Router. File-based routes compile to routeTree.gen.js; Axios
centralizes API access and attaches the auth token; Zustand persists the session.
erDiagram
COMPANY ||--o{ USER : employs
COMPANY ||--o{ SURVEY : owns
SURVEY ||--o{ SURVEY_VERSION : versions
SURVEY ||--o{ RECIPIENT : targets
SURVEY ||--o{ RESPONSE : receives
SURVEY_VERSION ||--o{ RESPONSE : defines
RECIPIENT ||--o{ COMMUNICATION : receives
| Model | Responsibility (key fields) |
|---|---|
User |
Admin account: email (unique), passwordHash (bcrypt cost 12, select:false), role (admin/viewer), hashed reset/verification tokens. |
Company |
Workspace identity and default branding (primaryColor, secondaryColor, defaultFont, thankYouMessage). |
Survey |
Mutable metadata: status (draft/published/closed), unique publicId, isWhitelistEnabled, oneResponsePerRecipient, currentVersion, publishedVersion, soft-delete, captureMetadata (opt-in). |
SurveyVersion |
Immutable, locked snapshot of sections, questions, visibilityRules, navigationRules, and presentation settings. |
Recipient |
Eligible contact (phone/email), status (pending→invited→in_progress→completed/failed), isBlacklisted. |
Response |
Version-bound answers: surveyVersion (number), answers (Map of questionId→value), responseStatus, progress, navigation, mode (test/live). |
Communication |
SMS/email delivery log (channel, status, externalId). |
Integrity through indexes. Compound indexes back dashboard and analytics queries; a unique
{surveyId, version} index keeps version numbering consistent; partial-unique indexes enforce one
recipient per contact per survey and one submission per recipient (or hashed identifier) per survey
version — the "one response" rule is guaranteed at the database level, not just in code.
Question types (seven): short text, long text, single choice, multiple choice, dropdown, rating (5- or 10-point), date.
| Decision | Why, and the trade-off accepted |
|---|---|
| MongoDB + Mongoose | A survey's shape varies per instrument, which maps naturally to flexible documents and embedded sub-schemas rather than rigid tables. Trade-off: schema discipline moves into Mongoose validators and app code. |
| React 19 + Vite | One renderer serves editor preview, public response, and response detail; Vite gives fast builds and a simple static deploy. Trade-off: large editor/analytics bundles need later performance work. |
| Express 5 | A small, modular REST boundary fits a JavaScript-everywhere stack. Trade-off: validation/error conventions must be established by hand. |
| JWT bearer auth | Cleanly separates a static client from the API. Trade-off: browser token storage raises XSS impact (§6). |
| Netlify + Render | Independent static and API deployment is cheap and easy to reason about. Trade-off: Render's free tier cold-starts add first-request latency. |
| Hand-authored OpenAPI 3.1 + Scalar | A machine- and human-readable contract, testable against the router. Trade-off: maintained by hand (guarded by a coverage test). |
The same editor and renderer drive two survey experiences from one stored definition: a
single-question flow (single_page), one question at a time, and a grouped, section-based
layout (multi_step), where related questions appear together. Because presentation is a property of
the schema (settings.presentationMode / isSectional) rather than hard-coded markup, adding a
question type is a renderer change and changing the layout is a configuration choice — the app is never
rebuilt per questionnaire. The same QuestionRenderer powers editor preview, the public response page,
and the response-detail view, so a question looks and behaves identically everywhere it appears.
Branching logic lives in two parallel modules: server/utils/logicEngine.js and
client/src/lib/utils/logicEngine.js (the client version is a superset that adds preview helpers).
Both share a recursive condition evaluator (all/any/not plus operators such as equals, in,
gt, contains, exists), visibility resolution (opt-in "show" rules, subtractive "hide" rules),
section navigation with jump/terminate/skip actions and circular-jump detection, and helpers
that compute which questions are answerable and required.
The client uses these rules to decide what to show; the server uses the same rules to decide what it
will accept. On submission the API re-runs the engine, computes the visible-and-required set, and
rejects a response (HTTP 422) if a required answer is missing or if an answer is submitted for a
question that should have been hidden (except in test mode). The client's display logic is therefore
non-authoritative — a respondent cannot bypass validation from the front end.
Every edit mints a new, locked SurveyVersion (currentVersion + 1); publishing marks which
version is live by setting publishedVersion = currentVersion. Each Response records the numeric
surveyVersion it was answered against, and the uniqueness indexes are keyed on that version. Editing
a survey after it is live cannot retroactively change what an earlier respondent saw, and a re-published
version lets a recipient answer the new version exactly once.
Express routers isolate product domains; Joi validates requests at the edge; controllers enforce
business rules; Mongoose adds persistence-level validation and indexes; shared middleware returns a
consistent envelope ({ status, data, paging }) with request IDs. Services wrap S3 uploads, branding,
HTTP, and email/SMS, degrading gracefully when optional credentials (SMS, AWS) are absent.
Controls in place: bcrypt password hashing (cost 12), JWT expiry, SHA-256-hashed password-reset tokens with a 10-minute expiry and no account-enumeration on forgot-password, an auth-endpoint rate limiter (10 attempts / 15 min), Helmet, layered Joi + Mongoose validation, tenant-scoped queries, hashed respondent identifiers, whitelist/blacklist checks, one-response constraints, rich-text sanitization, request IDs, and hidden production stack traces. Metadata capture is survey-level opt-in.
Honest gaps (verified in code) — the release-blocking items:
- CORS accepts all origins —
cors()is called with no allow-list. - The general API rate limiter is present but disabled — only
/api/authis throttled. - JWTs are stored in
localStorage— readable by any injected script;HttpOnly,SameSitecookies would reduce XSS token theft. - Role authorization is not enforced — an
authorizehelper exists but is not wired into any route, so aviewercan reach admin endpoints; "admin only" is documentation, not enforcement. - The development auth bypass is on by default outside production — safe on Render
(
NODE_ENV=production), but any non-prod deploy must setDEV_BYPASS_AUTH=falseexplicitly. /api/healthreports the database as "connected" as a literal rather than checking the live connection.
Privacy. These controls do not by themselves prove legal compliance. Kenya's Data Protection Act 2019 includes rights of access, correction, and deletion [verify — ODPC, 2024]; a sensitive deployment still needs a lawful basis, notices, retention/deletion rules, processor agreements, and any required impact assessment.
Accessibility. The UI includes labels, ARIA attributes, focus treatment, contrast utilities, and semantic primitives, with a keyboard-focus test scenario. WCAG 2.2 covers identifiable errors, instructions, keyboard use, and exposed name/role/value [verify — W3C, 2023]; full conformance requires automated and manual audits of complete journeys.
Unit tests run on Node's built-in test runner (no extra framework); end-to-end coverage uses Playwright with Allure reporting.
| Suite | Files | Tests | Result |
|---|---|---|---|
| Client unit | 11 | 73 | passing |
| Server unit | 14 | 44 | passing |
| Total unit | 25 | 117 | passing |
| Playwright e2e | 24 specs | — | cover critical journeys |
Unit tests cover the logic engine (branching, conflicts, randomization, section skips), question and
response validation, progress save, preview submission, recipient upload and derived status, analytics
export, and survey cloning. Six server tests are an OpenAPI coverage suite that fails the build if
any of the 52 annotated route operations is undocumented, if a documented operation has no route, if a
$ref fails to resolve, or if an authenticated operation omits bearer security — so the API docs
cannot silently drift from the router.
CI gap (stated honestly). There is no .github/ workflow: nothing runs the 117 tests, lint, and
build automatically before deploy (FR-11). The intended pipeline runs on every pull request:
flowchart LR
Commit[Push / PR] --> Install[npm ci]
Install --> Unit[Unit tests]
Install --> Lint[ESLint]
Install --> Build[Production build]
Install --> Spec[OpenAPI validation]
Unit --> Gate{All pass?}
Lint --> Gate
Build --> Gate
Spec --> Gate
Gate -- Yes --> Deploy[Netlify + Render]
Gate -- No --> Block[Block merge]
The client is a static bundle on Netlify; the API is a Node process on Render; MongoDB
Atlas backs both. Configuration is committed as netlify.toml and render.yaml.
flowchart LR
Repo[GitHub master] --> NB[Netlify build]
Repo --> RB[Render build]
NB --> CDN[client/dist on CDN]
RB --> API[Express API]
CDN -->|VITE_API_URL| API
API --> Mongo[(MongoDB Atlas)]
API --> S3[S3 assets]
API --> SMS[SMS provider]
| Concern | Netlify (client) | Render (API) |
|---|---|---|
| Build | npm run build in client/, publish dist (Node 22) |
npm install at the root |
| Start | static | node server/server.js |
| Health | — | /api/health |
| Routing | /* → /index.html (200) so TanStack Router owns deep links |
— |
| Key env | VITE_API_URL (baked in at build time) |
MONGO_URI, JWT_SECRET, FRONTEND_URL, AWS/SMS |
VITE_API_URL is compiled into the bundle at build time, so changing it requires a fresh deploy.
NODE_ENV=production on Render hides error stacks and disables the development auth bypass. Render's
free tier can sleep, delaying the first request (~50 s cold start); a production SLO needs an always-on
instance or an accepted cold-start budget, plus monitoring (health, 5xx, latency, DB, SMS, S3), log
redaction, and tested backups.
What the project achieves. SurveyFlow meets the central goal of the brief: it identifies a real feedback-management problem, implements a modular full-stack solution, deploys it publicly, and backs it with meaningful automated tests. Its strongest original decisions are the schema-driven builder with two presentation modes, server-side revalidation of survey logic as a trust boundary, versioned publishing, and integrated recipient and analytics workflows.
Priorities.
| Priority | Work | Acceptance criterion |
|---|---|---|
| P0 | Add CI + repair lint baseline | Unit, lint, build, and OpenAPI checks pass and are required on every PR |
| P0 | Harden auth and network boundary | Origin allow-list, general rate limits, cookie-based sessions, enforced roles |
| P0 | Complete privacy controls | Notice/consent, retention, export/access/deletion, DPIA where applicable |
| P1 | Establish accessibility evidence | Automated + manual WCAG 2.2 AA audit with critical defects closed |
| P1 | Isolate and run integration/e2e suites | Repeatable test environment and published reports |
| P1 | Observability + recovery | Dashboards, alerts, tested restore, defined RTO/RPO |
| P2 | Performance | Bundle budgets, lazy loading, load tests |
| P2 | Product value | Email invitations, templates, multilingual surveys, granular roles |
| Time | Content and demonstration |
|---|---|
| 0:00–0:50 | State the problem: feedback is valuable, but authoring, eligibility, distribution, version integrity, and analysis are fragmented |
| 0:50–1:30 | Introduce target users, value proposition, and research evidence |
| 1:30–3:00 | Log in, show the dashboard, open a survey, and explain the schema-driven editor |
| 3:00–4:10 | Add a question and demonstrate validation, section structure, and conditional navigation |
| 4:10–4:50 | Preview, publish, and explain immutable SurveyVersion snapshots |
| 4:50–5:50 | Show recipient upload/whitelisting, invitation options, and one-response controls |
| 5:50–6:50 | Open the public link, complete a branch, save/resume, and submit |
| 6:50–7:40 | Show analytics, response details, and export |
| 7:40–8:40 | Present the architecture and cloud-deployment diagram |
| 8:40–9:25 | Show test evidence, API documentation, and the health endpoint |
| 9:25–10:00 | Close with limitations, P0 roadmap, and the product outcome |
A recorded walkthrough of the running app (login → dashboard → answer & submit a survey → settings →
analytics) is included at docs/surveyflow-demo.mp4 — narrate over it to cover the plan above.
Prerequisites: Node.js 20+ and MongoDB (local mongod or Atlas).
git clone https://github.com/Ericokim/SurveyFlow.git
cd SurveyFlow
npm install
npm install --prefix client
cp .env.example .env # then fill it in
npm run dev # API + client togetherUseful commands:
npm test # unit suites (client + server)
npm run test:integration # server integration suite (needs a reachable MongoDB)
npm run test:e2e # Playwright
npm run build # production client build
npm run lint --prefix clientUpdate this report when scope, architecture, model, routes, security posture, test results, deployment targets, or public URLs change. Test counts should always come from a fresh run. Do not commit credentials, production exports, recipient files, or reports containing personal data.
Real, verifiable sources only. Confirm each before submitting; replace any you did not read.
- Mavletova, A., & Couper, M. P. (2016). Grouping of items in mobile web questionnaires. Field Methods, 28(2). [verify DOI]
- Office of the Data Protection Commissioner, Kenya. (2024). Personal Data Protection Handbook. [verify]
- World Bank. (2021). World Development Report 2021: Data for Better Lives. [verify]
- World Bank. (n.d.). Service Delivery Indicators. [verify]
- World Wide Web Consortium. (2023). Web Content Accessibility Guidelines (WCAG) 2.2. [verify]
Product tools referenced for comparison in §1.1 (not academic sources): Google Forms, Microsoft Forms, SurveyMonkey, Typeform, Qualtrics.
| Checkpoint expectation | Evidence in this report / repository |
|---|---|
| Identify a real-world problem | 1–2: fragmented feedback workflows, tool comparison, users, value proposition |
| Short literature review | 1.2 and References |
| Complete system architecture | 4 diagrams, data model, pipeline, technology decisions |
| Clean, modular, scalable implementation | 5 and the repository layout in README.md |
| Security and interdisciplinary design | 6 |
| Testing and CI | 7 — verified results and the documented CI gap |
| Public cloud deployment | 8 and the verified public links |
| Professional 2,500–3,500-word report | This document |
| Ten-minute presentation or demo | 10 plus docs/surveyflow-demo.mp4 |