How to Evaluate Technology Like an Architect
(Without Being One)
The Problem
You’re in a vendor pitch. The slides are beautiful. The demo is smooth. The salesperson keeps saying “seamless integration” and “enterprise-grade.” Your team is nodding along.
But something feels off. You can’t put your finger on it — you’re not a developer, not a systems architect — but your gut says this won’t work the way they’re promising.
Here’s the thing: You don’t need to be an architect to think like one.
Architects don’t have magic powers. They just ask different questions. They’ve seen enough systems fail (and a few succeed) that they’ve developed pattern recognition for what actually works versus what sounds good in a conference room.
This post is about those patterns. Not so you can build systems yourself, but so you can evaluate them the way architects do — and make decisions you won’t regret six months later.
The Framework: Four Pillars of Architectural Thinking
When architects evaluate technology, they’re running a parallel assessment across four dimensions. Not in sequence — all at once, constantly cycling through these lenses:
1. Boundaries → What connects to what, and where do things break?
2. Constraints → What are the real limits, and who’s pretending they don’t exist?
3. Change → What happens when (not if) things need to evolve?
4. Failure → How does this break, and what breaks when it does?
Most tech evaluations focus on features and cost. Architects focus on these four pillars because they predict whether the system will actually work in your environment — not just in the demo.
Let’s break down each one.
Pillar 1: Boundaries
Core question: Where does this system end and everything else begin?
Every system has edges. APIs. File formats. Network protocols. User directories. The question isn’t whether those boundaries exist — they always do. The question is whether you understand them before you commit.
Why this matters
In a 2023 manufacturing deployment I worked on, a plant wanted to connect their MES (Manufacturing Execution System) to a new analytics platform. The vendor demo showed beautiful real-time dashboards. Seamless, right?
Except the MES exposed data via a legacy SOAP API, updated every 15 minutes. The analytics platform expected REST, updated continuously. “Seamless” required building a middleware layer, polling the MES, transforming the data, and pushing to the analytics platform. Not impossible — but definitely not “plug and play.”
The technical team knew this on day one. Leadership found out six weeks later when the project was “90% done” and stalled.
Questions to ask
“What does this system need to connect to, and how do those connections work?”
Not “can it connect?” but “how does it connect?” Big difference.
“Where will data live, and in what format?”
Cloud? On-prem? Hybrid? Who owns the schema? Who pays for egress?
“What happens at the edges when things don’t match?”
Version mismatches. Protocol differences. Timing issues. How are gaps handled?
“Who owns the integration, and do they have the context to maintain it?”
Third-party SI? Your team? The vendor’s “professional services”? This matters.
Architect mindset: Boundaries aren’t obstacles to overcome — they’re design constraints to understand. The earlier you see them, the better your decisions.
Pillar 2: Constraints
Core question: What are the actual limits, and is anyone pretending they don’t exist?
Every system has constraints. Performance ceilings. Scalability thresholds. Compliance requirements. Vendor lock-in. Licensing restrictions.
The difference between a good tech decision and a regrettable one often comes down to whether you understood the constraints before you hit them.
Why this matters
In manufacturing, constraints are often invisible until production. I’ve seen plants implement “unlimited scalability” cloud platforms, then discover their factory network’s 10 Mbps uplink couldn’t support real-time data sync. The platform was fine. The constraint was elsewhere.
Architects hunt for constraints the way detectives look for motives. Not because they’re pessimistic — because unaddressed constraints become crises later.
Questions to ask
“What are the performance limits at scale?”
Not “can it handle our load?” but “what breaks first when load increases?”
Response time? Throughput? Storage? Cost?
“What compliance or regulatory constraints apply, and does this solution fit within them?”
Data residency. Audit trails. Retention policies. Industry certifications.
“We’ll handle compliance” is not an answer. “Here’s our SOC 2 report” is.
“What happens when we hit a limit?”
Graceful degradation? Hard failure? Exponential cost increase?
The answer reveals a lot about system design.
“What are we locked into if we choose this?”
Proprietary formats? Vendor-specific APIs? Long-term contracts?
Not all lock-in is bad — but it should be a conscious trade-off, not a surprise.
Architect mindset: Constraints aren’t failures — they’re reality. The question is whether you’re designing with them or against them.
Pillar 3: Change
Core question: What happens when (not if) things need to evolve?
No system stays static. Requirements change. Teams grow. Regulations shift. The vendor releases a new version. Your business pivots.
Architects evaluate change the way engineers evaluate stress — not whether it will happen, but how the system responds when it does.
Why this matters
A automotive parts manufacturer I worked with deployed a custom ERP module. It worked great — for two years. Then they acquired another plant. The module assumed a single-site model. Adapting it required rewriting core logic, breaking integrations, and delaying the acquisition benefits by nine months.
The system wasn’t wrong when they bought it. But it was rigid. Nobody asked “what happens when we grow?” because growth felt abstract. Until it wasn’t.
Questions to ask
“How does this system handle versioning and updates?”
Forced upgrades? Breaking changes? Backward compatibility?
“Cloud means always up-to-date!” can mean “your integrations break without warning.”
“What’s the cost (time, money, risk) of modifying this later?”
Open APIs? Extensibility points? Or hard-coded assumptions?
The easier it is to adapt, the less regret later.
“If this vendor disappeared tomorrow, what’s our exit plan?”
Can we export data? Migrate workloads? Or are we trapped?
Not paranoia — just contingency planning.
“Who on our team understands this well enough to support changes?”
Vendor-dependent? One internal expert? Documented and transferable?
Truck factor matters.
Architect mindset: Change is inevitable. The question is whether your system absorbs change or resists it.
Pillar 4: Failure
Core question: How does this break, and what breaks when it does?
Everything fails eventually. Servers crash. Networks partition. Bugs ship. Vendors go out of business. Users do unexpected things.
Architects don’t ask if systems fail — they ask how, and whether the failure mode is acceptable.
Why this matters
A Midwest manufacturer implemented a cloud-based production monitoring system. Looked great. Then their internet went down for four hours during a shift.
The system said it had offline mode. What it actually had was a 15-minute local cache. After that, operators were blind. Production didn’t stop — but efficiency tanked because nobody knew machine status in real-time.
The vendor wasn’t lying. But “offline mode” meant different things to them and the plant floor. One conversation about failure modes would have caught this.
Questions to ask
“What happens when this system is unavailable?”
Graceful degradation? Full stop? Cached mode? Manual fallback?
How long can you operate without it?
“What dependencies does this have, and what happens when they fail?”
Network, power, third-party APIs, authentication services, databases.
Single points of failure are fine — if you know they exist.
“How do we know when something is wrong, and how quickly?”
Monitoring? Alerts? Or do users call the help desk first?
Detection speed determines response speed.
“What’s the recovery process, and who owns it?”
Automated failover? Manual restore? Vendor support ticket?
Know the RTO (Recovery Time Objective) before you need it.
Architect mindset: Failure isn’t a design flaw — it’s a design input. Plan for it.
Putting It Together: Evaluation in Practice
Here’s how this plays out in a real evaluation:
You’re considering a new shop floor data platform. Instead of just asking “Does it have the features we need?” (everyone does), you cycle through the pillars:
Boundaries: “Our PLCs are 15 years old and use Modbus. How does your platform handle that?”
Constraints: “We’re in medical device manufacturing. Where does data get stored, and how do you handle 21 CFR Part 11 compliance?”
Change: “If we add a new production line next year, what’s involved in scaling this?”
Failure: “If your cloud service is unreachable, can operators still see machine status locally?”
None of these are gotcha questions. They’re clarifying questions. And the way vendors answer tells you as much as the answers themselves:
Good answer: Specific, honest, sometimes acknowledges trade-offs.
Bad answer: Vague, deflects, over-promises, pivots to features.
What This Isn't
This framework isn’t about:
Becoming a systems architect. You don’t need to design the system — just evaluate it intelligently.
Catching vendors in lies. Most vendors believe their pitches. The issue is often misalignment, not malice.
Avoiding all risk. Every choice has trade-offs. The goal is informed risk.
This is about asking better questions so you make decisions you won’t regret.
The Meta-Lesson
The most valuable thing architects bring isn’t technical expertise — it’s systems thinking.
They see connections others miss. They anticipate change. They assume failure and plan accordingly. They understand boundaries.
You don’t need a CS degree to think this way. You just need to ask the right questions, listen carefully to the answers, and trust your gut when something doesn’t add up.
Because the best defense against bad technology decisions isn’t more technical knowledge.
It’s better questions.
Start Here
Next time you’re evaluating technology — whether it’s a SaaS platform, an IoT deployment, or a new analytics tool — run it through the four pillars:
Boundaries → What connects to what, and where do things break?
Constraints → What are the real limits?
Change → What happens when things evolve?
Failure → How does this break, and what breaks when it does?
You’ll be surprised how quickly unclear answers surface. And that clarity — before you sign the contract — is worth everything.