AppTrajectory

Software Selection Methodology: How We Evaluate

Define essential requirements, compare evidence and trade-offs, test a representative workflow and document a defensible software choice.

Choose software against the work it must do and the responsibility you can sustain. AppTrajectory uses a requirements-led selection method. We explain why an option fits, why it may fail and what evidence could change the recommendation. A comparison need not end with a purchase or a numerical winner.

Stuart Kerr Spindlow is the Founder, Publisher and Principal Reviewer of AppTrajectory.com. He leads the publication’s editorial direction, evaluation framework and review standards.

A selection sequence you can reuse

  1. Define the outcome, users, decision owner, current baseline and constraints using the requirements checklist.
  2. Separate essential gates from optional preferences. Give each essential an acceptance check and owner before shortlisting.
  3. Choose between approaches before brands: improve the current process, configure an existing system, buy shared software or build a custom solution. Use build versus buy.
  4. Compare viable candidates against the same workflow, roles, data and cost horizon. Record evidence and unresolved questions in a matrix.
  5. Run a bounded pilot on the actual plan and roles you would buy. Test exceptions, integrations, prohibited access and usable export as well as the happy path.
  6. Document best fit, poor fit, compromises, ownership and switching costs. Choose, reject or defer; preserve the reason and the evidence that would trigger reconsideration.

A reusable comparison matrix

Selection criteria and the evidence to request
CriterionEssential or optional?Evidence or acceptance checkTrade-off to record
Workflow fitEssential for the agreed outcomeRepresentative users complete the task and an exception caseExtra steps, manual reconciliation and usability
Access and data requirementsEssential where the approved boundary requires themTest the intended roles, prohibited actions and recovery routePlan restrictions and ongoing administration
Integration requirementsEssential when the workflow depends on another systemCheck direction, fields, identifiers, failure alerts and retry behaviourConnector charges, API limits and responsibility for faults
Implementation complexityEssential if time or capacity is a hard constraintName setup, migration, training and support ownersConfiguration flexibility versus maintenance burden
Total ownership costEssential against the agreed budgetCompare supplier charges, labour, overlap and exit over the same horizonSeat bundles, usage growth and costs excluded from estimates
ScalabilityEssential for committed growth; otherwise a preferenceTest or verify expected users, records, concurrency and required controlsHigher tiers, operating effort and uncertain future demand
Portability and switchingEssential for required records and exit conditionsExport and reconcile records, relationships, attachments and permissionsLost history, retraining, overlap and vendor dependency

For each candidate add: result (pass, fail or unknown), source or observation date, applicable plan/version, evidence location, limitation and decision owner. An unanswered essential stays unknown. A feature tick or a high aggregate score cannot turn it into a pass.

Three kinds of evidence

What an evaluation statement establishes
EvidenceWhat it supportsIts limit
Stuart’s recorded hands-on observationWhat Stuart actually observed under the stated conditionsGeneral product use does not prove a task-specific test or comparative result
Verified documentationWhat an identified primary source states on its check dateA supplier description does not prove behaviour in your account
Editorial analysisA reasoned interpretation of requirements, evidence and trade-offsA recommendation is not a measured performance result

Illustrative examples explain a method; they are not customer outcomes. Calculations distinguish verified inputs from assumptions. Product-specific statements retain their source-check dates; a later editorial update does not imply every supplier fact was rechecked.

When weighted scores help

Use the evaluation scorecard only after the essentials pass and the alternatives have comparable evidence. Its 0–5 scale has defined meanings and user-defined weights; the formula and completion rules are visible on the tool page. These are your assessments, not AppTrajectory’s vendor ratings.

Agree weights before examining the totals. Explain why each priority matters, then vary the most uncertain weights or inputs. If a reasonable change reverses the order, describe the result as sensitive and investigate the deciding criterion. A qualitative matrix is often stronger when the evidence is sparse or options solve different problems. Do not score historical, policy or procedural pages simply to create a number.

A decision tree before purchase

  • Does the current process meet the essential requirements? If yes, require a specific gain that justifies changing it.
  • Does a candidate fail an essential requirement? Reject that configuration or revise the requirement explicitly with its owner.
  • Is an essential unknown? Request evidence or pilot it; defer the decision rather than assume a pass.
  • Do several candidates pass? Compare the useful differences, complete ownership cost and implementation burden.
  • Can the chosen option be operated, recovered and exited by named people? If not, simplify the solution or choose another approach.

Why buyers choose the wrong product

Common mistakes are choosing a brand before the workflow, treating optional features as essentials, testing a premium trial while buying a lower plan, overlooking integration failure, pricing only seats and assuming exports preserve everything. Test the actual configuration and the requirement most likely to rule it out.

Record the recommendation and its conditions

A useful conclusion states: choose this option if these requirements and responsibilities apply; avoid it if these constraints apply; prefer this alternative when this condition changes. Record limits, unresolved evidence, the owner and when to revisit the choice. Use the cost calculator for explicit estimates and the migration checklist before a switch.