Why Great Construction Leaders Don’t Always Look Great on Paper | Mark Smith, Ph.D.

Rishi Srivastava (00:41)
Today our guest is Mark Smith. Mark, welcome.

Mark Smith (00:45)
Thanks for having me. Looking forward to this.

Rishi Srivastava (00:48)
First section is on your background and why construction. What pulled you into focusing specifically on construction and engineering leadership? And what patterns did you see that made you commit to this niche?

Mark Smith (01:03)
Yeah, no, good question. It’s one folks ask me a lot. So I have a background in organizational psychology. So I got a PhD about 20 years ago in it. And soon after I started working in HR consulting. And really it was one of my first big assessment projects was with a home builder back in Houston. And then right after that I did

some big assessments for some commercial construction companies. And since then, I’ve done lots of companies, lots of industries. And the truth is, I think I just like working with the construction folks. I get engineering. Those people, it sort of feels like my people. And I’m married to a civil engineer, so that probably helps too.

Although my joke always is she’s only a civil engineer when she’s happy with me.

Rishi Srivastava (02:01)
Actually, working with people you like is so important.

Mark Smith (02:04)
Yeah, yeah. And you know, it’s funny, I worked with HR folks, I worked at the Society for Human Resource Management, and it was just like, man, I like the operations guys and things. And it was like, I could work with the HR people, but I always felt like I fit in more with this other group.

Rishi Srivastava (02:25)
You’ve assessed close to 1,000 candidates. What are the most common ways people look strong on paper but struggle once they are leading in the field?

Mark Smith (02:36)
Yeah, you know, this question of looking strong on paper is an issue that comes up time after time. And so I think I’ll address it in a couple of different ways. A lot of the people looking strong on paper are the ones who have like a long history of the right job titles, for one. And for two, they’ve got a good

like college degree, they’ve got a master’s degree or something like that. And what I’ve seen is the years of experience in the right job titles and level of education don’t really provide much information about how someone comes across in the assessments that I do. So if somebody you know, they’ve got a great work history,

Maybe they’ll come across really strong in the assessments, maybe they won’t. If they’ve got no college degree versus a master’s degree, maybe they’ll come across better or maybe they won’t. I haven’t seen much of a relationship there. And frankly, I trust my assessment results more than, again, years of experience, the right job titles, a college education.

Those things are important to some extent, but when it comes to evaluating candidates, I’d much rather rely on the results of the assessments versus making some assumptions based on how someone comes across on paper.

Rishi Srivastava (04:02)
interpersonal skills matter so much.

Mark Smith (04:05)
Yeah, they really do. you know, this is something that I will ask folks regularly when I’m working with it. When people succeed or fail in this job, what does that look like? And so often, you know, the descriptions of people that comes down to the soft skills, adaptability, communication.

you know, leadership, influence, those are the things. And when I start listing off those things, those aren’t the things that are taught in college. And maybe you pick up those things throughout your career, but some of the times people just don’t. Maybe they never got the right feedback or something like that, and they spent years just being a jerk or just being a pushover or whatever. then…

in which case years of experience, that’s not a useful indicator of much.

Rishi Srivastava (05:01)
Exactly. You’ve testified before Congress about skills first hiring versus degrees. How do you explain that idea to a contractor who’s never questioned traditional hiring before?

Mark Smith (05:16)
Yeah, so I think first, you know, we should describe what skills-based hiring is. And it can have different names, skills-based hiring, skills-first hiring, competency-based hiring, but it’s really putting the relevant skills and competencies for the job first and pushing things like education degrees and years of experience.

much further down the list. And so basically it’s saying if somebody doesn’t have a college degree that doesn’t disqualify them from from most jobs or at least it shouldn’t. And you can come at that from from different angles but you you bring up the know the testifying before congress that was such an interesting experience because the political left and the political right were in complete agreement that

We shouldn’t put as much weight on college degrees when it comes to hiring people as we do the far left person the far right person they could come and agree on this topic and It was funny after you know, we testified and it was a long hearing The person who was coordinating it he came up to me and he said, you know mark it never goes like this He says it’s

always contentious. Like the Republicans and the Democrats, they never agree. Like they agree on this topic. And it’s probably, I should say that I’m not bashing college degrees. In fact, I’ve got a 16 year old daughter and we’re starting the, what does the next step look like for you and is it college and…

Rishi Srivastava (06:35)
Mm-hmm.

Mark Smith (06:51)
And it’s probably college. But that doesn’t mean that somebody who doesn’t go to college shouldn’t be qualified for a lot of different jobs. In fact, that’s what I’ll always say that your qualification for job should be based on your skills and your competencies. And I don’t care where you got those things. If you got them from being on a college campus and learning from professors,

then that’s fine. But if you got them from, you know, watching videos and learning them from YouTube, well, maybe that’s fine too. It’s not up to me to care where the skills came from, where the competencies were developed. I just care that you have those. And when you see people being forced into the route of going to college, it’s not just that, it’s…

It takes so much time. Like a chunk of somebody’s otherwise earning careers is spent in college and you’re spending so much money to go to college. know, a lot of people, it’s a four plus year endeavor that they’re spending six figures on. That’s a lot of time and that’s a lot of money. And, you know, I don’t know that there’s the right bang for the buck when it comes to that.

Rishi Srivastava (08:13)
down.

The next section here mark’s on skills first hiring in practice. If a contractor says, we hire on gut feel, what’s the simplest skill first process you’d recommend that still feels fast and practical?

Mark Smith (08:31)
Right. Yeah. So when it comes to hiring, a lot of times companies will just, you know, and they’ll just admit to it. Yeah, It is gut feel. And again, I have a PhD in organizational psychology. And one of the things we’ve done is for a series of different ways of evaluating candidates. We’ve done what we call

validation studies and so you can look at how well somebody does on an interview or an assessment and there are different ways of scoring these different things and then you can get information on their job performance and then you can correlate the two you can correlate you know does how well somebody comes across in an interview does that predict who’s going to be a better job performer and that

type of study is what we call a predictive validation study. does the assessment results actually predict job performance? And what the results over time have shown is that an unstructured interview is kind of worthless when it comes to predicting job performance. Because an unstructured interview, you’re treating your different candidates differently.

there is no comparing apples to apples. It’s always apples to oranges because you’re asking different questions to your different people or you’re asking maybe the same questions, but you’re evaluating them differently. And one of the things we’ll always say again, from organizational psychology is that a process can’t be valid. It can’t be accurate.

if it’s not first reliable. So if your process isn’t repeatable, it’s not going to be accurate. And so in the interview process, we just have to implement structure. We got to make sure you’re treating all your candidates in the same basic way. You’re asking them the same questions. You’re evaluating them in the same basic way. And your questions and the ways that you’re evaluating them are based on

the skills, the abilities, the competencies that you need for the job. So we start by looking at the job and the organization and say, okay, these are the things that the person needs to be able to do or know or whatever. And then you design some questions to assess those things. And then you just go through that process with the different people. And so basically every company currently uses an interview process. And so the easiest thing is to,

change a current process and so that okay let’s update your interview process and make it structured and then after that there are additional things that we could do as well and that’s and one of those additional things is what i spend more of my time doing these days

Rishi Srivastava (11:21)
Yeah, know, hiring is tough. A lot of times it’s our own biases that we go with when we’re hiring people.

Mark Smith (11:28)
Well, and those biases a lot of times can come out when there is no structure in the process. And so if you’re just left to your own devices to create interview questions on the fly or just decide, OK, I like this person or I don’t like this person, these things that you shouldn’t be using to evaluate people can just creep in. And so some forethought, some planning can can help just just keep those those.

biases out of the process.

Rishi Srivastava (11:55)
For project managers and superintendents, which leadership competencies actually matter most and which ones do companies consistently overvalue?

Mark Smith (12:07)
Yeah, and this goes back to some of the things we had talked about before, which is the soft skills. Those are the things that often matter. So what does it take to get fired around here? A lot of the times it’s, the guy was a jerk or he just couldn’t control anything on his projects. And a lot of times, you know, that has to do with adaptability and that has to do with communication.

Those are the things that are typically undervalued in a process because those are the things that are often trickier to evaluate in a hiring process. The things that are often overvalued are specific experiences and like specific technical background. And so, oh, this person uses the project management platform that we use already. Oh, that’s really good.

Well, is it though? Because at this point, a lot of the project management platforms are pretty similar to each other. You know, maybe the layout is different and some has a bit more functionality on this piece versus another piece. But if you’ve used a project management software or platform or application, you’re probably able to pick up a different one. And even if you’ve used the same one,

you’re probably going to go through training anyways because you might have used it in a previous company a little bit different than this company uses it. And so you’re already going to go through training on it anyways. And so why even care about that? As long as you know that they have some ability to pick up these things. Does it really matter that they have the right software in their work history? The truth is it probably doesn’t, but

Those are easy things to talk about and pick up in an interview and say, yeah, he used this platform. that’s great. I don’t know that it is.

Rishi Srivastava (14:06)
Yeah. Can you share a story where an assessment helped the company avoid a bad hire or promotion that otherwise would have looked like the obvious choice?

Mark Smith (14:18)
Yeah, you you just you come across all these different characters and types of people in these assessments that you’re doing. And it’s just funny how their personalities come out. I know one person a few years ago, we were evaluating there was for this company, it was a heavy industrial company and there was an opening at a senior director level, I think it was.

and we were evaluating some internal candidates. I don’t know if there were serious promotion candidates or not, but they just had us evaluating a series of these. And, you know, one of the guys was, he was a director. He looked okay on paper. And in the roleplay assessment that he was doing with us, I’ve done a bunch of these assessments. I kind of know what to expect.

I know what’s normal and accordingly I know what’s weird. And about, we had just started it. And he looks right at me during this role play assessment and he goes, why are you being such an asshole? And I said, we just started. Why are you calling me names? And it was his way I think of, you know,

putting himself as the alpha in the situation. But man, he came across as really aggressive and he just thought that was an appropriate way to act and an appropriate way to get the upper hand. I think he was just being a jerk. And my assessment report read accordingly and I’m happy that the company didn’t promote a guy who just

thought that that was an appropriate way to act and whether it was a real scenario or not a real scenario, you shouldn’t be acting like that at work.

Rishi Srivastava (16:04)
Personality matters. I like to be working with people who are good and competent.

Mark Smith (16:09)
Yeah, yeah. And again, when you ask the question of what does it take for somebody to fail here or what does it take for somebody to get fired from here, it’s usually it’s personality related. They just refuse to follow the rules. They knew what the rules were and they refused to follow it. You know, what are we going to do? Well, hopefully you’re going to avoid most of those before you put them in a position to have that happen.

Rishi Srivastava (16:33)
The next section is Assessments, Scenarios, and Hard to Fake Signals. Clients say your approach makes it much harder for candidates to just say what employers want to hear. What specifically makes your assessments harder to game?

Mark Smith (16:49)
makes your assessments harder to gain.

Yeah, so I think at this point we should walk through what my assessments actually look like because I think they’re different than what people anticipate when I start talking about them. So when people talk about assessments, especially these days, I think they’re often thinking of computer assessments. They’re thinking of personality inventories where

You have a list of a hundred items and for each item you’re rating it on a scale of one to five. This is like me, this isn’t like me. Or you have a little math test or a verbal comprehension test and you get a score based on that. Those are traditional assessments, those are assessments that have been used before. We can talk about the merits of all of them. But my assessments are different.

My assessments are very work related and they look at other things, not just and really not cognitive stuff at all. So it’s work related and so it is a role play assessment. So how it goes is we schedule a time with a candidate and at that time they’ll get an email from us which has

some materials and basically the materials are it’s your first day on the job, here’s the job that you have, here’s the type of organization, and here are a couple of situations that are going on. So it’s not very long, it takes you know seven minutes to to read those materials but they have 20 minutes to review those and we tell them at the end of 20 minutes you’re going to meet with somebody and we tell them who it is and different of these exercises you’re going to meet with a different person.

and basically say it’s your meeting so you can run it however you want. So they have 20 minutes to prepare after the 20 minutes they get on a zoom call with me or somebody like me and I’m playing a character at that point and they’re kind of playing themselves in a different situation I’m playing a character and there is no right or wrong you just see okay

there’s some information that they had, did they understand the information? Were they able to fill in the gaps in a reasonable way? And then what do they choose to do? How are they going to treat me? What does the conversation look like and feel like? You know, there are a lot of ways that they can treat this interaction. in rare instances, are we thinking about it in terms of right or wrong? It’s just

we allow them to do whatever they think is good and appropriate. And then after that 20 minutes, there’s the last 20 minutes is what we call a debriefing chat. And in the debriefing chat, we just talk about, hey, how do you think that interaction went? You know, what did you do to spend spend your 20 minutes preparing? You know, were there things that you would do different? So just just some things at the end about debriefing. And so the

interaction is very job relevant for construction engineering and very realistic. Like they’re just treating it as this is kind of a real situation. And so even if they prepare and they’re thinking at the beginning, okay, I need to sort of pretend I’m somebody that I’m not. The situation is so real that a minute or two into it, they stop pretending because it just feels real and

I’ve talked to so many people after the fact now, and literally everybody says, this is the thing that everybody says, it felt so real. It felt so real. I’ve talked to so many people and everybody, they’ll have different descriptions, but almost everybody verbatim will say, it felt so real. And the fact that it felt real made them sort of treat it as real. And even if they wanted to fake it somehow or game it as you say,

they’re starting something. I’m reacting. They don’t know how I’m going to react. So there is no way to plan it advanced to be somebody different than you are. You just, you, you have to be who you are because all these things are happening real time. There is no time to, to plan anything else. And so it, it’s, it’s realistic. And the fact that it’s just happening on the fly, people, people just can’t fake it. Maybe they can try, but that, that goes away quickly.

Rishi Srivastava (21:16)
Tell us a story about a candidate who interviewed extremely well, but whose behavior in a simulation revealed a leadership risk no one expected.

Mark Smith (21:26)
Yeah, yeah, yeah, the leadership risks no one expected. You know, there as I mentioned before, there are a lot of times, people, they’ll have impressive backgrounds. And then you’ll run them through the assessment and you’ll see somebody different than who you expected to see. So maybe six months ago, I was dealing with somebody, they were a previous COO at

a mid-sized construction firm and you go, okay, that’s pretty high up. They probably had lots of people reporting them. They had a lot of authority and they probably had some sort of, you know, expertise coming in. But I realized in the scenario that we were going through is they didn’t really know how to, I guess, run a meeting with a direct report and

maybe it was in their background they had other people do this but it was something I thought this person would be great at it would just be comfortable doing it would be very natural for them and as it turned out they just they didn’t understand how how to do that how to run a report with the direct or run a meeting with the direct report and

Again, I was kind of surprised. Would this have avoided a bad hire?

It was something that was clearly missing in their background that came out in their behaviors that again surprised me and I wouldn’t have assumed it based on based on his resume. Otherwise a nice guy. It’s just it feels like there was something missing and if they would have hired him for that open role I think it probably would have been a mistake. Some of the times people they get into positions in their career maybe before they’re ready.

And accordingly, they’ve got like some high level skills and then they’re missing some like lower level skills. And again, you assume that if somebody has got high experience that they’ve got all the sort of scaffolded skills that are built up to it. And that’s just not always the case.

Rishi Srivastava (23:33)
Like if you’re working for the same company for a really long time, you could get to very high up even with not having all the personality and technical skills.

Mark Smith (23:47)
Yeah, yeah, that’s right. And I guess I’m hearing these stories more recently than before is when mergers and acquisitions happen, those things all of a sudden get exposed. And so the person who has been in the one company for 20 years, now they’re in this sort of merged company. And the way that they’ve done things for years.

didn’t feel weird to them. And now everybody looking at it’s like, what the heck is Mark doing? Why is he acting that way? And the truth is maybe Mark act that way for years and nobody said anything or no one even thought it was weird. And the fact that, you you went through this merger and all of a sudden, you know, this, and maybe it’s not weakness, but this sort of hole, this lack of understanding of something.

got exposed in a way that you never thought of before.

Rishi Srivastava (24:44)
What’s the difference between assessing someone for selection, hiring versus development coaching an existing leader and where do companies get this wrong?

Mark Smith (24:56)
Yeah. So in my world, so the assessments that I do are similar, whether it’s for hiring or for development. So the actual assessments themselves are the same. The difference is in the report. So if I’m evaluating, I’m assessing somebody for hiring purposes,

The report isn’t usually for them, it’s for the hiring manager. so accordingly we’re talking about somebody in third person, we’re talking about, and you can talk about them in little bit more, I guess, black and white sort of clear ways because it’s not directed at the person. On the other hand, if it’s an assessment done for developmental purposes, the report goes to that individual and

it needs to be written in a way that is maybe not universally positive, but is developmental and it can be helpful for the person. And so if you’re talking to the hiring manager, you don’t really have to worry about that. If you’re talking to the person, you don’t want to be dismissive of them. You don’t want to be negative towards them. You want to be helpful because the process is designed to be developmental. And so

Rishi Srivastava (25:47)
Mmm.

Mark Smith (26:13)
So that’s the big thing is the tone of the report. And the other thing is in the developmental report, there is an additional end section, which isn’t descriptive anymore. Now it’s sort of recommending some things that somebody might do. It’s, you know, here are some books that might be useful or interesting to you, and here’s why. Or here are some training courses that you might consider.

Or here are some activities that you might engage in on the job that can help develop some maybe some holes that you currently have. And because a developmental report that doesn’t have that isn’t going to be terribly useful. We’ll often say like like insight without action is just silly. If you’re going to

give somebody really good feedback, but you’re not going to tell them what to do with it, then they’re going to, you know, what are they supposed to do with that? Okay, no, here’s what we’re seeing. Now here’s maybe what you might do with it. And, and, and so that, that’s the other piece that makes the developmental reports different than the hiring reports. What becomes interesting is a situation, which is kind of a hybrid of the two. It’s,

it’s the promotion candidates because you’ve got internal candidates for promotion. You run them through an assessment like this. And in one way, it’s like a hiring scenario because you’ve got to choose, you’ve got four people or something you’re evaluating and they’re only going to promote one of them. And so, you know, is this like a hiring situation? Yeah, it is like a hiring situation. But at the end of the day,

they’re still internal to the company. And so you can have a hiring plus a developmental situation. And that’s what a promotion kind of looks like because you might have a hiring manager report and a developmental report for the candidate based on a single assessment. the promotion situations are actually you might have two different reports based on a single assessment.

Rishi Srivastava (28:27)
somewhere, know, a critic who’s directly criticizing a person, giving them some feedback, but without giving them the action needed, it’s just not a good critic. You know, it’s hurt somebody directly and they haven’t told them how to improve.

Mark Smith (28:43)
Right. Yeah. It’s funny. I’ll talk to folks and a lot of companies will have like, maybe they’ll have LinkedIn learning or they’ll have, you know, some other sort of training resources internally. And often it’s you, they’ll tell these, these people, Hey, we’ve got these training resources and they’ll say, okay, great. What should I do? And

And they’ll say, find something interesting and, take a class on it. We’ll find something interesting is not a useful, you know, direction in terms of development. It’s okay. Based on who I am, what I bring to the table and where I want to go. What should I do? which is not just a, what training classes should I take? But you know, what, what should I do? And just to

to leave people on their own to decide. That’s poor form. I don’t know why people do it. It’s sort of a figure it out kind of thing. And what ends up happening is people take things that they’re comfortable with that they already like. And so it’s probably that maybe they’ll go through a course or they’ll read a book on a strength that they already have. Well, that’s not helpful.

It’s a strength that you already have, but they’re comfortable with it. And so they’re happy to go through this thing and they did really well and they get the warm fuzzies and they feel good about it, but it’s not useful. And so pairing the feedback with the direction of what to do. And I guess the other thing that I should add is the what to do is often not a book and it’s not a training course, it’s…

something on the job. It’s, you can read about leadership or you can take a class on leadership, but to actually lead a team, well, the leading the team thing, that will probably actually help you learn to be a better leader. And so, a lot of times companies will have, you know, these ad hoc committees or volunteer committees, and they’re always looking for people to help lead them. And if you’ve got a

a young guy who’s never led anything before and that’s something he needs to develop, well, you should probably tell him, find something to lead and lead it. Don’t go to a training class. Don’t read a book about leadership. Find something where you can actually do it. And then what’ll happen is he’ll do that and he’ll screw up the first thing that he does. And that will be the most important learning.

that he had was actually doing something and actually screwing it up and then cleaning up his own mess and then they’re like, okay, I’ll never do that again.

Rishi Srivastava (31:28)
Yeah, real world action taking. The next section here mark’s on mid-sized contractors and real world constraints. You focus on small and mid-sized construction firms. What leadership failure patterns show up most often in that segment?

Mark Smith (31:31)
Yeah, that’s right.

Yeah, it’s the smaller companies, know, that it’s the construction and the engineering. So often what you run into is that there’s no leadership development going on. It’s not there’s some leadership development or or they kind of do a halfway job of it. There’s typically just no attention to that, no resources being expended.

And accordingly, you can have somebody he’s been in leadership position for 25 years or she’s been in a leadership position for 18 years and has never gone through any sort of leadership development or personal development at all. They’ve taken some training and that that’s it. And so the general rule coming in is there isn’t anything going on. Now, the larger companies, yeah, they may have like a whole leadership academy and

and things like that. the smaller ones, you know, especially the smaller growing ones, you know, they’re resource constrained. They’re just trying to get by. The the owners of the company will be the first to say, we would have loved to do that. But there was no money for that. There was no time for that. We were flying by the seat of our pants. And, you know, this was just something that, you know, maybe it was on the priority list, but it was low enough on the priority list that it just never got done.

And again, that’s the rule. That’s not the exception. Most small mid-sized construction and engineering companies will say something like that. They’re going to turn their attention, provide resources to technical kind of things, and not these soft skill kind of things. And certainly, they would rather not deal with a psychologist of any kind.

Rishi Srivastava (33:31)
Yeah, it’s tough too. mean, when you’re a small company, there’s just so many priorities and these soft kind of skills, they never seem to be important.

Mark Smith (33:43)
Yeah, that’s right. And you know, I get it. I understand.

Rishi Srivastava (33:48)
Let’s go to the next question. In the smaller companies, HR is often compliance-driven, while real decisions sit with owners. CFOs are ops leaders. How do you design assessments that actually work in that reality?

Mark Smith (34:02)
Yeah, and so the companies that I deal with, so if they have like 50 employees, there’s probably no HR person at all. There would be some sort of admin person who takes care of lots of things. If it gets a little larger, if it’s 100 person, 200, they’ll probably have an HR person. But as you say, that person is probably more rules and compliance driven.

And so mainly we’re dealing with, the owners, the, leaders, the, operations folks. And that’s one of the reasons that I made sure that the assessments that I’m using and recommending are, very, and I’ll refer to them as face valid, but are very obviously job related. They look like the job, they feel like the job it’s clear.

Rishi Srivastava (34:50)
⁓

Mark Smith (34:51)
that they’re appropriate for it. When you’re dealing with HR folks, they may be just fine with, these are computer tests of verbal reasoning and mathematical understanding and a personality assessment. When you hand those types of assessments to an operations leader, they’re gonna say, what the hell is this? I’m not using that.

And that’s another one those things. I don’t blame them at all. Yeah, the what the hell is this is actually a reasonable response to, you know, here’s a computer assessment or a set of assessments. When they see and especially when they experience the types of assessments that I’ll do, they’d say, I understand this. This feels right. This can provide me good information and this can provide…

a way to differentiate between people. And when it comes to hiring, especially at the end of a hiring process, and so at the end of a hiring process, you’re just considering a couple of people, it’s usually very difficult to decide, why candidate A, but not B or C? You know, why hire her when these other two guys are seemingly good as well?

you know, what information should we use? And then I say, I have this job relevant, kind of scenario based assessment process that will feel realistic to these candidates. And it will provide you with a way to get additional information that can legitimately differentiate between these people. And a lot of times it’s not even good or bad differentiating. It’s who’s going to be a better fit here?

because as you know, there may be somebody that you’re considering for a job and you’d say, that’s a great fit in this company. He’d be a terrible fit over there. Or she’d be a wonderful addition because she provides this that this company doesn’t have. But she’d be a terrible fit at this other place because when it comes to final candidates, it’s usually not good or bad. It’s…

who is a good fit here and where can we get legitimate information to find out if somebody is a good fit? Because hiring is not about checking boxes and it’s not about counting years of experience. It’s about predicting the future in a lot of ways. Because it’s not even who’s right now. It’s…

a of years down the road who will be the best fit for us in the future. Not today, in the future.

Rishi Srivastava (37:32)
so much about future in your head.

Mark Smith (37:34)
It’s

a tall order when it comes to doing this well. And when people think of hiring, they’re just like, we’ll just open the hiring thing that we use. And the first question they’ll say is, how many years of experience and what kind of background and what kind of education do they need? And those are just the default screening things that they use. I’m just going to say, yeah, we need to rethink this altogether.

Rishi Srivastava (38:00)
Can you share a story where a company had to choose between two strong candidates and how the assessment made the decision clear?

Mark Smith (38:09)
Yeah, so was just one that I was thinking about just now. was for a senior director kind of role. And there were two final candidates, good candidates. And we put them through our director assessment. And I mean, in that scenario, both of them, they

You know, they did well to the extent that I’m evaluating them and sort of doing well or doing poorly, but they were different. They addressed the situation completely different. And it’s not up to me to say, she did better than him, or you should hire him and not her. It’s to say, this is the person that you’re going to hire if you hire her, and this is the person you’re going to hire if you’re going to hire him.

It’s not up to me to say who’s the best candidate. It’s up to me to describe them accurately. And so it’s that kind of situation. And so, yeah, in this case, know, one was very relationship oriented and one was very task oriented. And something like that, that’s not a right or wrong. Yeah, you kind of need a balance of both.

But if somebody naturally veers toward the relationship side or they naturally veer toward the task-oriented side, it depends on what the company is like, what the company is looking for, what the situation, what the clients demand. And again, that’s not up to me. What is up to me is to describe them accurately and to use an assessment process that will get to an accurate description. And so…

That’s what we were able to do. I don’t even know who they hired, but they probably would have been okay hiring either one of them. But the point is that whatever way they went, one, they were able to make an informed decision, but two, they were gonna get a very different person depending on who they hired. Some of the times you’ll run people through a process and they’ll end up looking very similar. And that’s accurate too. I can’t promise that.

that will provide this huge wedge between people. But I can provide a job relevant, realistic assessment that will provide good information.

Rishi Srivastava (40:22)
well described. The last section here mark’s on ROI, AI, and what’s next. If a CFO or owner asks, what’s the ROI of better leadership selection? What outcomes do you point to? Schedule, margin, turnover, safety, claims, or something else?

Mark Smith (40:43)
That’s a good question and I’ll probably answer it in a way that reveals that I’m not a real salesperson and I’m not in marketing. I’m a PhD organizational psychologist. I wish I could point to studies about ROI of this type of thing or what types of outcomes are most affected by this type of thing. And I’m not going to make up studies that don’t exist.

I know that meta analysis of these types of assessments have been done and they’ve shown that they’re good at predicting job performance. So they’re good at showing who’s going to be a better job performer or worse job performer. But on the specific metrics, I’m not sure. Again, I’m designing these to look at soft skills. And so I think the metrics that are most affected by soft skills and so

that’s going to be probably less on the financial side and more on the relationship side is probably what I expect. people would argue, well, know, everything is about relationships and the relationships will affect the financials. I just haven’t seen any actual studies that will show me, you can expect a 20 % ROI. can’t say that. I’m not going to say that. And I’m not going to pretend

that’s an accurate number that I can come up with.

Rishi Srivastava (42:02)
So I was selling construction accounts payable automation software to an IT leader. And he said, like, any time you have to actually go on and do an ROI calculation, you’ve already lost the battle. The ROI usually is so obvious that you don’t have to do any calculation.

Mark Smith (42:17)
Right. Right. And, and,

know, I’ve, I’ve gone down those route before and, know, there are, there are like equations that economists will use about how to determine this. And once you go into the equation and you go, wait, why is that number 25 % and, and

and they’ll be like, well, it’s best not to just question that. And you go, but the entire, your entire argument rests on this number. And as far as I can tell, you just made that up. then, well, no, everybody in the field uses that. And you go, okay, well, you know, maybe for consistency sake, you can use it, but let’s not pretend that that’s a meaningful number when you just made it up.

Rishi Srivastava (43:01)
Yeah, last question is looking ahead three to five years, how do you see AI changing hiring and leadership development in construction and what new risks should leaders be aware of?

Mark Smith (43:15)
In the last couple of years, there have been a lot of new assessments coming out, which at least claim to be based in AI. And these assessments get a little squirrely just because AI is a little bit tough to put in a box right now. And it’s hard to understand what’s going on. so in fact, just, I think yesterday I was looking at there was a new company that just announced and

They had these basically job relevant role play kind of assessments, just what I’m talking about here. But you’re not dealing with a person, you’re dealing with an AI that’s sort of a text based kind of thing. you’re writing in what you were going to do and the AI kind of responds to you. And in the past, you would think that was algorithm based. And so that was a lot of if then kind of things.

And now with AI, there’s a lot more that can go into that. And it’s actually hard to control that AI to like keep it on task. And then once it’s done interacting with the person, and again, this is sort of in a tech space thing, it’s a big black box in terms of how the AI system is evaluating and assessing the candidate.

That being said, so I’m still incredibly skeptical of AI right today in this type of assessment. There are other assessments that AI is getting better at. They feel less job related than what I’m talking about. Five years from now, we may be to the point where there’s a fake me engaging with a person that feels real. It would no longer be sort of written text-based, but it would be verbal, but it would…

It would be the same thing. So at some point, the AI might replace me. But there are a lot of subtle things that you sort of need to evaluate because it’s not even what the person says there’s tone of voice, you know, just sort of, how does this he or she come across that AI still is having a hard time.

accurately evaluating facial expressions, of micro responses and that type of thing, which people can naturally pick up on, but AI systems are having a hard time. So five years from now, this discussion is probably different. Three years from now, I don’t know that three years, but about five years, I think that that’s about right where somebody like a fake me, a fake assessor, a fake role player,

you know, could actually do the job. But right now, that’s a bit of a wild card. It’s hard to control and it wouldn’t feel realistic right now either.

Rishi Srivastava (45:57)
There’s a company that has developed an AI interview taker. you instead of interviewing an employee, that AI is going to interview, come to the interview and answer your questions. It’s just so weird.

Mark Smith (46:10)
Yeah, it’s funny,

I know, and this has been around for a couple years, sort of like the, they’ll refer to it as an AI interview, but you’re just responding to a question that’s written it and you’re talking to a camera and then the AI system is evaluating it after the fact. But to me, that’s not an interview because in interview, there has to be some sort of interaction.

There has to be a give and take between the interviewer and the interviewee. And those AI systems where you’re just answering a static question and you’re talking to a camera and it’s recording you and it’s evaluating it. Yeah, to me that might be useful, but it’s not an interview.

Rishi Srivastava (46:50)
Mark, thank you so much for your time. It was a good conversation.

Mark Smith (46:54)
Alright, great. Thank you.