Grading Colleges?
Yesterday I walked down the kitchen and the Gotham Gal was reading the paper and she looked up and said “Obama wants to rate colleges. I think that’s a bad idea”. I thought about it for a second and said “hmm. I need to think about that”. We are both on the boards of trustees of higher education institutions (strangely not ones we attended in both cases). So I think its important to state upfront that my thoughts on this topic are not in any way related to the schools we are affiliated with.
The New York Times has a debate on this topic on their website today. I read all the views and understand the pros and cons. Here is where I come out.
1) Colleges and Universities should be held accountable for the outcomes they produce, particularly for students whose tuition is paid by taxpayers in the forms of grants or below market loans. If the taxpayers are footing the bills, we deserve to know where we are getting a return and where we are not.
2) The Federal Government is the wrong entity to do this. The chances that they will mess this up are too high to put them in charge of this.
3) We should peer produce this data, wikipedia style. Something like ratemyprofessors.com. Students who get federal or state funding for some or all of their education should be required to submit information to such a service. Students whose educations are not funded by the government should be encouraged to opt in and report. The data should not include personally identifiable information (although I recognize that it would theoretically be possible to reverse engineer it if someone really wanted to do that). Most importantly, we should collect data on how the students do in their careers so we can measure real outcomes. Grades and graduations rates are nice. Earning power five and ten years out is even more important.
4) This would be a completely open database. All data would be available to be analyzed via open and public APIs, like the early days of the Twitter API. This would allow many third parties to analyze the data. There is no one truth, but if you triangulate, you can get closer to it.
Building this would not be hard. Making it the standard would be harder. It should be a non profit like Wikipedia is so everyone can and will trust it and work with it. And the federal and state governments should adopt it, support it, and require participation in it for those who are benefitting from taxpayer dollars.
If Obama really wants to rate Colleges and Universities, and I think we should be doing that, then this is the right way to do it.
Comments (Archived):
is it Obama, or is it the top universities wanting this? moating strategy.
WOW. Alumni Associations would be gaming this somehow. But we should know what works and why about education.
they game the US News and World Report rankings alreadyin an open platform where everyone can analyze the data it would be easier to detect the gaming that will obviously be attempted
There are business schools in the world that signal to their students the finer points of filling out surveys so rankings go higher.
I hate that when the car dealer emails you that post-sales survey and hints they expect Excellent ratings everywhere.
That is the epitome of bad service. Worse so because they think they’re doing good service!
yep. that industry hasn’t changed too much for years.
Our company’s website is the largest for student content on the Internet, http://colleges.niche.com (originally College Prowler). We have tens of millions of opinions from students already. We tackle this in some instances but it’s hard to get good coverage across thousands of colleges. It’s also difficult to get good normalized data but we’re working hard on solving this already.
We liked College Prowler when we looked at schools. The problem back then was it wasn’t always current enough (on some less popular schools)-so some of the info was stale. But, we learned some things and it was another perspective on some schools we were looking at.
We’re getting better all the time. Again, hard to keep 8000+ colleges completely up to date. A lot of times we’re at the mercy of the schools who don’t do things in a standard way.This year 40% of all college bound high school grads in the US have a College Prowler account. We’re making progress.
Yup, I didn’t mean to make my comment a “diss” on College Prowler. We used it and it was another perspective outside of the media ones, opinions of friends, and wherever else you could glean info.
My daughter is headed off to college in the fall. FWIW, we used College Prowler to check our impressions of the campuses we visited. For the school she decided to attend, our impression from the campus visit and the general consensus on College Prowler were well aligned. Which I take as a good sign.
Thanks, Chris! Appreciate the feedback. Glad it worked out.
Awesome
We’d be happy to partner with Dept of Ed on this. Reminds me to ping Jim Shelton right now … Thanks for this post.
Looks like a great launch pad for targeting the issue.
We’re happy to help.
Wow. Amazing that getting found via a AVC post was more effective than a Google search.[I previously searched on Google for that topic, and your site wasn’t found. How is your SEO? ]
What was your search query? Our SEO is far from great. That being said we still get 80-90% of our traffic from Organic. 1.5M uniques a month.
I think I tried “college rankings us”. It could be some basic structural tweaks. I haven’t dived into it. Run an SEO tool on it, and it might diagnose some quick fixes.
we’re nowhere to be found on that query. we used to be on the first page right after US News but maybe something is structurally wrong right now. I’ll look into it. Thanks.
It’s true – “give a dollar so it counts towards alumni donation participation” is not uncommon.
I totally agree. They most certainly do game the rankings already.
US News and World Report rankingsThat, and others who do the same, worst thing that every happened to higher education.
Publishers have traditionally “ranked” universities, e.g. “U.S. News Rankings” in the US and “Maclean’s Rankings” in Canada, and these are wildly used by prospective students, no? Your proposal would yield additional data and insights.Maybe you could crowdfund the realization of this project on CrowdRise.
I look at how the rating systems worked out at public primary and secondary schools and give up.I look at services like Rate my Professor, and while they have some good comments, just like Yelp you get some crazy anonymous comments as well.Then you have the issue if you just worry about outcomes we really shouldn’t fund many liberal arts degrees, or degrees for those that are marginal students. As somebody with engineering and business degrees I should feel good about the first, but I don’t because diversity is what makes the U.S. strong. As somebody that feels more people should get trade degrees I should feel good about the second, I don’t.Finally you come to the tough part of the issue, and that is understanding outcomes of human beings that you are teaching. That is extremely variable. You could have been the best high school French teacher in the world (and I think I had some really good ones) but guess what??? My outcome and learning French sucked. Just didn’t feel like doing that. Who is to blame? Me or the teacher.Now rating how many administrators there are and how much they get paid?? All for it. One of my pet projects is to look at how administrators say “we are going to have to cut education!!” You look and the administrative staff has tripled. The cut they talk about is the cut in the rate of increase, and the threat they make and keep is to actually cut teachers.Which is my last subject. Any profession that works on the tenure system where you can’t fire people is crazy. Could you imagine if I was phoning it in and the worst you could do to me is nothing?
There are many reasons not to do it. By we should anyway
If its Fred’s ranking system or some sort of wiki, ok, if it has anything to do with the compensation of administrators no.
The bureaucracy to run education is the thing that is sucking all the money out of education. It takes $200B to manage $500B of govt grants-education is probably worse. At one public college in Michigan, the economics dept had 10 profs ten years ago and one or two administrative assts. Now they have more admins than professors.
We agree completely. It happens in all forms of government but education at all levels is stricken with the disease.My favorite anecdote comes from a guy with whom I co-founded a company. We both believe you can’t fix the world, but you can fix one part of your world. If everybody did this and fought the fight we would be better off.His kids school announced cuts to many programs including art and music. He could not see how this could happen given the total size of the budget had increased so much over the decade. Now this is a guy that probably can’t refer to somebody in band or who paints something other than walls in any way that would not be construed to be a politically incorrect slur.But as we say he hates waste more than #$^% band #$%^. The amount of waste he found was incredible. The vitriol he faced in rooting it out, exposing it, and then eventually taking over the school board and killing it was epic. Again though we were lucky that nothing motivates him more than a good fight.
Yup. When you root out waste in government, you usually are rooting out someone’s power base or patronage. Almost impossible. The character assassination alone stops people. If you don’t cheerlead all the time or go with the flow, people start sniping.
The vitriol he faced in rooting it out, exposing it, and then eventually taking over the school board and killing it was epic. Again though we were lucky that nothing motivates him more than a good fight.Great story. Ironic that in school you are taught to get along with others and to not, in a sense, rock the boat. Share your toys.So to answer: We both believe you can’t fix the world, but you can fix one part of your world. If everybody did this and fought the fight we would be better off.Most people need affirmation that they are well liked and that others will agree with what they are doing (and either join with them or applaud them).So they are not willing to take on battles that will make them unpopular and/or that won’t give them any good feedback from their peers.Every now and then you get someone like your friend “that nothing motivates him more than a good fight.”. I’m sort of like that as well. Not that I get motivated by a good fight (I’m motivated without a fight) but I’m not running a popularity contest and quite frankly don’t care (as much as others) about what anyone thinks.Same reason I do what is best for my stepkids because I’m not trying to be their friend but their parent (but in an odd way they seem to really like this about me..)
Why did the bureaucracy happen
Thomas Jefferson and the founders can speak more eloquently on that than I. But, once government starts expanding it’s like the kid in Willie Wonka, it never stops. Stigler showed how private industry even helps government expand through regulatory capture. Eventually it overwhelms everyone and everything. I think it’s important to note the Constitution doesn’t give us any rights-it protects us from encroaching govt. Bill of Rights outlines our rights, which also protect us from govt. The $200B cost was found by streamlinksoftware.com. Salaries, pensions, offices et al.
Tenure for college profs isn’t really a good job for life. If a prof can’t get a job better elsewhere, then why pay him (her, here and below) more? So, a tenured prof who quits working finds that his income doesn’t keep up with inflation, his teaching load increases, his office location is poor, his travel budget is low, his office furniture is old, his budgets for course graders and laboratory equipment are low, he doesn’t get a secretary, and he is treated with contempt. Even a tenured prof still has to please his colleagues, department chair, and college dean.Also, usually tenure is not so easy to get: By the time a prof has tenure, usually he has long since proven his dedication to building a great career. And it’s easy enough and routine to get rid of an unwanted tenured prof — make him miserable and offer to write him a really good recommendation to anywhere else.A good research university very much needs each prof to pull their own weight in research results, fame for the university, successful Ph.D. students, and external funding. For that last, main sources are NSF and NIH, and there the competition is very tough.
Glad to hear that, so it should be no issue to get rid of it. I am going to use your argument.
Dunno about education but feedback in general is honestly platformless today.Every call we make to a bank, every email to a service is now followed by some old school survey, print or voice.Cave paintings for a connected world.This is an area of huge potential and pent up need both on the expression side and on the access to useful information.
Every call we make to a bank, every email to a service is now followed by some old school survey, print or voice.Apparently the CEO’s of those companies don’t take those surveys if they did they would know how annoying they were. (Talking about the “Forsee” surveys that pop up everywhere I go..)
A democratic society rests on the concept of voting our politicians. A meritocratic society is simply an extension of this idea. Good service needs to rise to the top and bad service needs to fall away.And not just universities. Post offices, banks, plumbers, you name it.I’ve dedicated my last three years to making this happen in a clear and open way.
Earning power is certainly one metric.However qualitative feedback though harder to assess is meaningful.Some of the world’s greatest contributors are on Ramen diets for a while. DaVinci, Einstein.Shakespeare, Dickens Columbus illistrate my point. There are many others some possibly reading this now.
Then let’s capture that too
+100 for attitude
Obama’s idea is incredibly dumb. It’s mostly politics to distract the country from the various pitfalls of his administration.Agree that if one were to try and create a system to rate colleges-a P2P system is the only way. However, because of all the randomness, the data is really messy. What factors should be weighted higher than other factors?Gary Becker did a lot of research on human capital. He found that children from two parent homes that became college graduates did better than high school grads or high school drop outs.Still, is going down one tight path a predictor of success? No. It hinges on the individual taking what they learned, using the resources they have and working for it. Better to instruct kids not to create a mountain of debt, and to major in things that will get them ahead.
Agreed, data is way to “messy” to create any meaningful list or tight grouping of measurements.
I imagine the biggest challenge here would be to get a good, clean data set. As James notes in his comment it’s not just quantitative feedback (e.g. earning power) but qualitative as well.The intent is good: if you’re applying for financial aid that’s at least partially supported by the government it would be good to know that you’re using that money to attend a reputable university and not some drive-by-night diploma-printing operation. If you’re the first person in your family to go to college I imagine that it’s hard to discern that not all higher ed experiences give you the same value and thus hard to decide where to invest in your education.
We should take this a step farther and look at the same data for high school. The measure of a high school is not SAT or exit test scores, or the percentage of students that go on to college. It’s the percentage of students that are productive citizens after high school. If half the graduates of a high school are on public assistance 5 years later, that high school is failing. I imagine there will be a strong correlation between that sort of thing and test scores, but let’s the end result instead of the proxy.
If half the graduates of a high school are on public assistance 5 years later, that high school is failing.Well we already know that in many poor areas the high schools are pretty sucky.Because who wants to work at a high school where your life is in danger and where the student body is getting pregnant at 14.My father in law worked at a high school in Harlem and it was so dangerous he got paid more and was able to retire early. And the stress caused him plenty of health problems. That he still deals with (as a retired teacher that is actually in his 60’s).The teachers at my private school on the other hand used to have us to classes in their apartments on the campus. One teacher’s father was a cabinet secty in the Ford Administration. Many of them had IVY or other “good” degrees. While they could have taught at the “failing” schools unless they were saints why would they? (Separately the pay was less than in public education..)
I feel like glassdoor could do this tomorrow if they could figure out the incentives.
I think someone could do an anonymous study using Facebook data. Most profiles list the university the person attended, their local area of residence, and many list the job/job title. I don’t believe the quality of the university I attended is directly correlated to the job I have, my income, or where I live. But if someone thought these were proxies for educational quality, then looking at FB’s big data could get you a rating system.
The current rankings encompass more than educational feed back. Rankings are subject to a weighted scale that includes alumni donations, sports standings, and a host of other input along with career placement and earnings. I agree government ratings are vulnerable to the potential political landscape of the times.
I want an unbiased platform for the grading of politicians.
Good luck with that. Hope you have the patience of Job.
I know, we could vote every 2, 4, or 6 years and the ones with the most votes we would keep.Everyone loves THEIR politician who brings home pork to their district. Everyone HATES the politicians from other districts who bring home pork.
Oooh, Nice. We could call it Democracy. Has a nice ring to it.
A Republic, if you can keep it.
From the WaPo yesterday: http://www.washingtonpost.c…
We should think about moving more towards assessing the students based on knowledge they should have attained in their chosen career field. Europe has moved to this model. Community Colleges follow this model to obtain Federal funding. Colleges would resist tooth and nail however.
“The Federal Government is the wrong entity to do this. The chances that they will mess this up are too high to put them in charge of this.”This line can and should be applied to piratically everything that the US Federal Government does.
Jordan Goldman has collected a lot of different types of data — not all — to help incoming students evaluate options at http://www.unigo.com. If one were to scrape data from Linked In and backward map career paths of college graduates (job placement and career path being just one — not the only — outcome measure) and then marry those findings with analyses of Unigo’s data, you could begin an input-output assessment of colleges that would form the basis of an empirical college “grading” system.
that’s a fantastic idea Diana. we should find someone to work on this as an alternative to what the government wants to do
Great idea.To me, providing data for students can be complementary, but not substituted with, “grading.” http://www.bestmatchforyou.com has some of the data that perhaps you’re talking about—from PayScale, AACSB, the Martin Prosperity Institute, etc.—that provides some insights into graduates. We only work with MBA programs, but it helps students look at the information THEY find important to make decisions. It doesn’t tackle the Dept. of Ed’s desire to better allocate their funding but does deal with the issue of helping students make better-informed decisions. It’s a first step in this big picture.
Another useful source of feedback would be employers, who might convey how well-prepared recent graduates they’ve hired are, on an institution-by-institution basis.It would be difficult to ask the right questions and get good responses, but if scaled, users would have insight into how capable graduates in a given major and field are (or aren’t), compared to their peers from others schools.
Academic freedom & accountability for performance ( teaching, financial, even professional conduct ) have never made good bedfellows.My alma mater is in the soup on this front right now ( google University of Saskatchewan & watch the fun ).Your idea is a solid work around.Splitting institutions into research & teaching groups makes even more sense to me. That way, they can be held accountable to focuses outcomes.
I totally agree to the concept of rating universities, but I think there are 3 main criteria to evaluate a university:1. How soon on average a graduate gets a job and how much was his starting salary compared to the national average (thing like coop opportunities might also be considered)2. How much ROI research grants have generated for the companies/agencies offering the grants3. Which is going to be the hardest to measure: one of the criteria university professor s are measured by is research papers, but usually these research papers are a lot of noise more so than a problem with an actionable solution, a really effective measure would be to calculate the ratio of noise. Any paper that helped a government agency or a private company solve an industry problem is not considered noise, anything else does.Finally, how the students felt about the university, professors, environment, etc which in my opinion helps a future student decide between two universities when all the above is the same.
Different schools have different strengths, approaches and programs. Rating suggests standardization when our economy and society expressly needs the exact opposite. Honestly, I think it’s a terrible idea and will be a costly exercise for very little value.
I was thinking the same- that you’ll need to get into the granularity level of departments and schools in order to make this as meaningful as possible.
maybe rating is the wrong word to usecompanies do 360s on all of their employees and that turns out to be incredibly valuable in terms of managing a large organizationyou can’t manage what you don’t measure
Measurement is not ranking and list creation and then I would get worried about what we are measuring on. It is how much money people make? What about contribution to society? What about research and development? It becomes a very difficult thing to be able to an apples to apples comparison on unless we go to simpler quantifiable things (ala job status and $$) which to me are rarely a measure of true success.
I have been looking at this through the eyes of SMBs recently and ran across this company, JuvodHR.com that I kinda like.
We agree because you appear to me to be an analog thinker. Many of the people supporting the rankings are digital “manage by numbers” thinkers who aren’t into the nuance of the situation the work in broad strokes. I run into this with computer programmers and the like who look at things in a very black and white manner and don’t see much gray in anything.
To me that’s not analog vs. digital — that’s quantitative vs. qualitative — and it’s always been my belief that numbers are easily manipulated as and are anything but black and white – the ‘gray’ of them is higher up the funnel and just harder to detect. Before i had my career in marketing/digital I was an environmental planner who focused on evaluating Environmental Impact Assessments where 90% of what i did was find the bad assumptions underlying mathematical/quant models. (and the irony is that i suck at math 🙂
you can’t manage what you don’t measureThat’s Meg Whitman’s mantra. Unfortunately this issue is much larger than that and using numbers can also create unintended consequences. Because then people go toward the goal of meeting the numbers or gaming them.Separately Whitman’s not doing to well at HP. No surprise. Because there is no low hanging fruit of opportunity there as there was at ebay which was more like riding the bull of increased internet usage.http://time.com/118927/meg-…
You’re picking on Meg, poor Meg, poor, poor Meg. We should all give all our sympathy for poor Meg as she fires another 10,000 HP employees because the HP management doesn’t know what to do with those people. Wait! Hold on! Meg IS HP management!Of course she doesn’t know. There was nothing in her background that suggested she would know.’Management’, heck. ‘Management by the numbers’? Heck. She can leave such things to a COO. Instead, HP needs a leader, to say what the heck are the next objectives of HP. There are companies that can pick good objectives — Samsung, Apple, Google, Amazon, BMW, and more.But, no, you are being so cruel to Meg, so cruel. How can you be so cruel? Don’t you have any sympathy at all for all the suffering she must go through as she fires another 10,000 HP employees?
Now just as I was ready to confuse Sandberg with Mayer [1] I find/remember that there was a story about Sandberg being considered (could be a rumour) to run Disney. Once again what’s the basis for that? Two different situations as far as skill set and capabilities. Almost rises to the level of how people elect politicians. Amazing what a halo can do to someone’s resume.Anyway Mayer at Yahoo (what I was going to talk about here) is really similar. Different skills needed there than whatever got her to be who she was at Google.Separately is there any precedence for this type of behavior at mature old line companies?Like back when some guy said “If you can find a better car, buy it”.http://en.wikipedia.org/wik…My god it sends shivers up the spine:https://www.youtube.com/wat…[1] No I’m not picking on women. Tim Armstrong is perhaps another example.
Sheryl Sandberg’s only COO at Facebook; no doubt Zuck runs the place.Notice that I didn’t mention Marissa Mayer at Yahoo or Ginni Rometty at IBM. And I left out Patricia C. Dunn (charged with four felonies) at HP and with ‘qualified’ admiration from Tom Perkins.For Ginni, long IBM was “not an electronics company or a computer company but a marketing company”, and maybe that’s what she’s still trying to pursue although she doesn’t have much left to ‘market’.HP was Bill Hewlett and Dave Packard; they actually did real things, starting with an audio oscillator and growing into some super nifty electronic devices, some of the best medical and laboratory instrumentation, a good leadership position in server farm and digital network management, and high end servers and storage boxes. But technology keeps moving ahead, and it’s crucial to keep up — Apple did; also Android, Amazon, Samsung, Intel, etc. HP didn’t Sorry ’bout that.Failure’s no fun. I know about failure: No way could I learn French in high school because the teacher went writing first, reading second, and speaking third, and it never stuck. In college the guy went reading and speaking first, more reading second, and writing third, with some rote memorization, and it still sticks.Likely Meg is a nice person, but apparently as the head of HP she just doesn’t have “what it takes”. Hewlett, Packard, Jobs, Page, Brin, Gates, Zuck, Bezos, did/do.
Agree on Hewlett, Packard, Jobs, Gates and potentially Bezos (get’s A+ for getting one over on investors, we will know for sure in several years if he cuts off the air supply or not of his completion.)Would strangle Gates for the frustration he put on the world with his OS bugs and annoyances but he already had a halo with me just based on pulling off the purchase of dos from the guy who hadn’t been around the block apparently.Page and Brin don’t agree. Were spoon fed the moves by adults on how to ride the bull. They learned well of course.Zuck gets real world credit for pulling a fast one on the winklevi (shows young promise and bloodthirstyness) but FB is a one trick pony he is riding. As you can see with a President money buys lot’s of people to help you make the right decisions.Jobs of course wasn’t nice to Woz and took advantage of him. Had he not had that personality most likely we wouldn’t have the Apple we have today.Separately, Patricia C. Dunn, well, like Nixon, she got caught. [1] You have to be careful when working with power tools, not get cocky and wear eye protection.[1] Kind of funny when you compare Nixon getting caught with some burglary to Obama in the hot seat over Merkel.
As I remember the Dunn story, she’d worked her way up from secretary to COB of HP. Amazing.Then at HP, some guy did some publicity interface with the press, like he’d long done, and Dunn got all upset over maybe he’d let out ‘material’ information that would cause shareholder law suits or some such. So, Dunn goes into over reaction, wacko mode and sends spooks after the guy and maybe some others at HP. So, she is determined to hire some ‘plumbers’ to stop the ‘leaks’. She goes paranoid, hysterical, and obsessive-compulsive, all symptoms of ‘anxiety’ disease, four times more common in women than men. She does so much with the spooks that she gets charged with four felonies. We’re talking wacko over reaction.Tom Perkins told her that all she had to do was just tell the guy not to do it again.But mostly she can’t be blamed directly for the HP problems because she was a ‘non-executive’ COB; still, a #1 job of the COB is to fire/hire the CEO where the direct blame should go.Yes, the Iacocca ad was terrific.
Forgot to add to my comment below (re: youtube of lee) that while disciples of Eric Ries are laughing at the commercial they may want to consider what they failed to learn in school or in actual practice about classic marketing and salesmanship. And how important it will end up being when the shit hits the fan in their operation as they loose their available runway.
exactly 🙂
Yes, and what gets measured gets done, often with deleterious consequences.
“You can’t manage what you don’t measure.” Another reason to it’s a bad idea. The implication is that the government should manage college institutions. Look how well they’ve done with K-12…. any questions?
Ratings could be dynamic. For example, not just a 1-10 scale based on some algorithm or criteria. Could take into account different strengths, approaches and programs.
It more depends on how the data is interpreted, though you’re likely right that it will be that it will lead to people saying we should then control for X, X being the most “successful,” and reduce Y, not being as successful.
Specialized programs (I’m looking at you, SUNY college of forestry) Would be really difficult to rate. What would a rating for a school that trains for say the park service and Bureau of land management be rated about.Fore those curious : http://www.esf.edu
I agree that it’s a bad idea but more because I don’t believe a non game-able system can be devised that will provide the intended value.If you are familiar with how schools are ranked in certain states (blue ribbon) and how the teachers and the entire system is re-engineered solely to improve those rankings you might agree. Not to mention this is happening in healthcare now as well. I hear about this with respect to my wife at her hospital. You actually get rated based on short term patient happiness. So the doctors tend to optimize toward getting those reviews in a certain direction. Not all of course but enough to make an impact and it’s the wrong type of impact in terms of health care outcomes.
The Federal Government is the wrong entity to do this. The chances that they will mess this up are too high to put them in charge of this.+1000I agree with holding colleges and universities accountable – especially when tax dollars are involved – but education is a very difficult thing to grade. Access to data for public analysis along with some kind of gathering of existing opinions ( another comment mentions Facebook data ) seems like a good idea, but agreeing on what makes a good college is difficult. People head to college for different reasons, to study different things, and we all have different biasses, interests, and perspectives. Those cloud our own judgement sometimes.The standard line we heard from students is that all NYU cares about is accumulating real estate, but most parents I know don’t share that opinion ( but we might have other negative things to say ).And I know in my own case … I want “success” for my children, and I expect that to include some measure of financial independence, but I don’t think “earning potential” is a reliable measure of what they have learned – and even if it were, there is no way to sort out how much credit for that should be given to their universities.
Schools should be rated. All schools; grade schools, high schools, colleges and graduate schools. The last people to ask about a school’s performance are TEACHERS! Professional teachers have a distorted idea of what constitutes good teaching and useful education. One nice trick would reduce student debt for every review filed each year after graduation with a sliding scale “paying” more for the review 5 years after graduation than in the first year so that we capture long term data.
Rankings would definitely be one way of hacking education but we need to expand it to include a robust ranking system for faculty, especially tenure faculty. Administration care about rankings while faculty are worried about peer recognition.If there’s a way to combine both so that any student getting any tax payer money has to report on every professor they have, and not just a end of semester survey, but legitimate skills assessment and what they learned, it could be the best of both worlds. Schools would have to be accountable for the faculty they hire and ultimately for the results (students with/without jobs) they produce.
Artists, scholars, social workers, teachers…. how does a bureaucrat measure an “outcome” for society where the goal isn’t making as much money as possible? Can a business school weigh the ethics and humanity inherent in business decisions, alongside the profits, when measuring the success of alumni? Quantitative analysis is destroying education.
I would suspect most people go to college for an economic “outcome.”
How about artists, scholars, social workers, teachers, to name a few. Scientists. These people aren’t in it for cash. There are far easier ways to make money.We can (continue to) gear education for capitalism, which will create strong disincetives within the system to produce a Dostoevsky, Foster Wallace, Ellison (Ralph, not Larry), etc. Sales reps work to the comp plan, as they say.
“Success is liking yourself, liking what you do, and liking how you do it.”― Maya AngelouMeasure that outcome. RIP
She died today:http://www.cnn.com/2014/05/…
If they had demand for a rating system, someone create a different one for them too…
Or as someone said”Imagination is more important than knowledge”and that’s hard to teach and hard to measure.
There’s this rating too from Find the Best http://colleges.findthebest…
2) The Federal Government is the wrong entity to do this. The chances that they will mess this up are too high to put them in charge of this.The above can’t be overstated in my book.
Fred, if you don’t trust the federal government to rate colleges without screwing it up, how do you trust it to do most of the rest of what it does?I’m not arguing that the Feds wouldn’t screw this up, but given the huge scope of the federal government, this college rating business would be kind of small potatoes compared to the rest. If you don’t trust them to do this, logically, you shouldn’t trust them to do much at all.
All universities are already rated in the eyes of the public and pretty much always have been (i.e. “Ivy League”)…but putting some cold-hard stats and numbers behind what each involves and produces so that we can start to compare apples to apples would not only be awesome…it seems long overdue in my book…
the top tier havethe middle tier less sothe lower tier not at all
it doesn’t look like the lower tier private schools and many of the mid tier private schools are going to survive
The Chinese, paying full tuition, have started to bail these colleges out:
What’s top tier?
Transparency is one problem of many in the higher education industry. And it’s unfortunate that students and taxpayers aren’t already demanding it. Unfortunately this alone will not solve the biggest problems. There are actions the government, and others, could take to that could be much more effective. For example, the government could stop making below market loans with taxpayer dollars, employers could stop demanding applicants have degrees and find a more effective means of evaluation, students could put more thought into their decision to pay such high prices for college (transparency like this would help), etc.Completely agree with your #2. Not to mention the classic moral arguments of forcing people to pay for something they don’t necessarily want.
I always found it odd that the institutions that create the greatest capitalists in the world act so socialist.I don’t know if rating schools is the way to go but doing something that reminds them students come first I am for.
Strangely enough I just started thinking and about this yesterday. I started toying with a search engine ranking approach at the department level. The first weighted element of the ranking will be based on the college text books used in each of core classes. The books will be first grouped into clusters based on academic rigor and that rigor will factor into the ranking of the backed to the department.
Take tax money out of funding colleges and they will automatically be rated exactly the same way Amazon, Barnes and Noble and Radio Shack have been rated…. by their customers’ wallets.
You mean amplify “the network effect” as a proxy for educational metrics ?
Sure if that’s part of the value that the institution offers, so be it.
by their customers’ walletsWell for one thing this is a long term playout it’s not a trip to Jamaica, a Bourbon or a toaster.And people have already figured out a few things and that was prior to all the garbage that US News et al foist on the public. What they have found out is that certain degrees and certain schools are just better and for whatever reason, right or wrong, open up doors and make sense. You know back in the day when parents would compare notes and figure out that “being a doctor is a good career” and “MIT is a good school” and all of that.Back in the day this was all anecdotal and in general from my experience has played out pretty much as was discussed way back in the 70’s and earlier for that matter.Only exception that I can find is law and that’s a recent phenomena. And the only reason for that was it became to easy to get a law degree and it became a fall back career for everyone and their cousin.
Agreed, seems like a solution in search of a problem.
Harvard was doing well long before they were flooded with tax money. One source of funds has long been wealthy people, e.g., Johns Hopkins, who thought that a good college would be a good idea. Students have rarely paid the full cost of college.
I think ratings are extremely helpful. They build trust on eBay and Uber, help me pick out good new restaurants on Foursquare and Yelp, and choose which tea kettle to buy on Amazon. The two hard parts are deciding what is important to measure/value and how to measure it.These are the kinds of things for which I always think it prudent to re-visit Gladwell’s spaghetti sauce story: http://www.ted.com/talks/ma…There is no perfect spaghetti sauce, only perfect spaghetti sauces.
why not blow up the entire framework for a better online-hybrid alternative.you are through it (i believe) with your children fred – but for folks like me who have little ones (4,2 and -3months) – the daunting prospect of even being able to afford a higher education coupled with its highly questionable ROI (unless in the 1%) has me hoping for some drastic disruption to the model of higher education over the next decade.
The problem at the top level schools is one of the most valuable things you get from the university experience is a network rather than an education. How does that get rated?
Networks are self fulfilling prophecies. They get better and better the more valuable the network
Yup. Every single current supreme court justice attended either Harvard or Yale (mostly Harvard).
Sure but that’s like a winner take all problem. People should stop thinking that they have to win some prestige and position game to be happy in life. There are plenty of places hundreds of thousands of spots down where people do very well with where they are in life.Separately if all Scotus are from Harvard/Yale, part of that is that when graduating with that degree you are more naturally launched on a trajectory of opportunity than one who attends the local county law school.(Which is another advantage of a degree from a good school you get better opportunities right out of school..)
It doesn’t get rated. And in fact none other than Malcolm Gladwell apparently doesn’t see the value in a big name IVY education. He is wrong. Which shows how dangerous it is to listen to people just because they are famous and well known.)By the way it’s more than a network that you get. (If I understand what you mean by network you mean “contacts”, right?)You get to attend school with people who have cleared certain bars and are serious about the education experience and about learning. And passionate about what they are doing. Maybe not in 100% of the cases (some always slip though) but enough to create a positive learning environment that goes way beyond what the actual educational material is.Not to mention that the halo from attending a good school is worth it’s weight in gold in opening doors. (Doesn’t mean they stay open and doesn’t mean a student will be a hustler enough to take advantage of them of course.)Here is one for everyone. Do you think that attending MIT helped Fred get his first job in VC? Of course Fred also hustled (as he has told the story) to get the job. But do you think that the MIT degree helped and was worth the extra money vs. a less expensive school? I do. Further, do you think that Joanne, when she started dating Fred and/or Joanne’s family was impressed by the fact he attended MIT? And later Wharton? And did the MIT degree (and his grades) help him get into Wharton? Of course it did.Regardless of whether you think that mattered (or whether Fred or Joanne would agree or disagree) I am firmly in the camp that it does matter in a high percentage of cases. Meaning if you take 100 Fred’s who went to MIT there is a way higher probability of a Joanne deciding to date Fred and Fred getting his first job as a result of that degree.That said instead of everyone whining about how they didn’t get into a good school or now are disadvantaged concentrate on what you can do and work hard.
I am right there with you. A crowd sourced review will produce the kinds of results that are also helpful as they can come with explanation. The point being that some lower ranked schools may still be the best for the student. A simple grade or worse from the government will be far less useful. Hope this can happen, I will support it
I like the concept of a ratings system, but basing it on outcome is tricky. For better or worse much of the benefit derived from college is through their “signaling” ability and not necessarily the difference in education quality. Its not always clear that “better” schools provide better education, only that they have picked the best students.
meeker this morning. frightening, broken.
I automatically discard information from anyone who tries to argue based on facts and figures which to me is always suspect.To wit:”USA ranks 4th globally”. etc. So what? Didn’t know this was some kind of contest. Assuming correct 4th isn’t so bad. This isn’t a football game.And who cares what the entire country’s rank is in math? Those figures include all the riff raff that we have (go through Camden, Newark on West Virginia) and that assumes whatever the ranking is (and the methodology is) is correct. I didn’t do particularly well in math (maybe average) and I certainly learned my way around numbers enough to earn a living.”1/3 of four year college graduates feel”. – This is total utter crap and bullshit. Who cares what they feel? When were they surveyed anyway? Maybe we should correlate what music they like as well.Golden girl meeker gets the STFU award this am.
Poor Mary Meeker. I suspect that she’s a nice person and am sure she works very hard. But she fell into some of the MSM junk think, smelly bait for the ad hook, about US K-12 education.When they say that US K-12 education in math is no good, I’m insulted and outraged. In a US public high school, I did plenty well in math, thank you, on any scale, SAT, Germany, Japan, China, Singapore, Finland, Russia, etc. My Math SATs were over 750 both times, and when I got out of college my Math knowledge GRE was 800. Those numbers put my math knowledge in late high school and college up in the ozone for the world. I’ve published peer reviewed original research in applied math in well respected, international journals of research — in those journals, my papers look just fine. Did I mention that the journals were international?It’s easy to run down US K-12 math: Just take averages and compare averages. Compare the US average with those of the ethnically homogeneous countries that did the best. Presto. Done. And that’s what much of the MSM like to do, if only to have some smelly bait for the ad hook.But, if we ‘control on country of origin’, then we get closer to comparing the schools themselves and not mostly just countries of origin. So, to compare schools, let’s compare US students in Minnesota of Swedish or Norwegian descent with students in Sweden or Norway; that way we compare the schools in Minnesota with the schools in Sweden or Norway. Uh, we were talking about comparing the schools, weren’t we? The schools, right?Then, similarly, let’s compare US students of West African descent with students in West Africa. We also get to compare US Jewish students with students in Israel: In one famous study, the US students did significantly better. Why? Hint: The schools in Israel actually have a lot of Arabian students!For more, controlling on country of origin, just what country does better than the US? Is there one? That is, is there a country X such that students in that country do better than students in the US with origin country X? Got a candidate for X? More than one? A country of significant size? If there is a country X, then maybe US researchers should discover that country and meet its better students at the award ceremonies in Stockholm.The 3 K background radiation? Let’s see now: That was Penzias and Wilson at Bell Labs in NJ. A theoretical explanation was from Dicke at Princeton, NJ. More generally, last time I checked, the US was still doing quite well in Stockholm. Amazing considering that US schools are so far behind and that schools in some other countries are so much better. Amazing.Why the confusion? It’s from people with zero ability at analysis or deliberate, and I pick the second. So, the deliberate goal is “no child left behind”, that is, not schooling or education but ‘social engineering’. To see some of the magnitude of the problem, see the segment on PBS ‘Frontline’ on what Michelle Rhee struggled with at the DC public schools.Could the ‘social engineering’ be successful? I don’t know; maybe. But a failure of a challenging goal in social engineering is not necessarily a failure in US K-12 schools. Uh, the schools can’t do it all, and the students have to do at least something!
Much as been written about the huge financial burden many students face post graduation, with many students facing a very low prob of ever paying off the debt they’ve accumulated.Who’s to blame for that–the university, the student or both? Caveat emptor certainly comes into play, as many students don’t fully consider at a pre-loan stage the prob of payback of their accumulated debt based on the earning potential of their intended profession (or set of possible professions). Students need to do more due diligence in determining whether university and/or a specific profession is a proper life choice, in the context of their current individual/family economic situation and their own future earning potential. Many don’t consider, and it likely will impact the quality of their lives for many years…if not for eternity.If such due diligence led to considerably more students not applying, you’d then quickly see a more aggressive response from university admin to change their biz model, with an eye more towards affordability. Perhaps many more universities would deploy a hybrid model–part in-class, part MOOCs–as an economic solution Bottom line: Those universities that don’t adapt, will go under….and perhaps a weeding out process isn’t such a bad thing.
I think the hardest part of picking a college is asking a 17 year old kid if he is willing to take out college loans (whether is be $20K or $120K). At 17, it is too young to really be making those decisions and if a student it lucky enough to have parents to guide them through the process it is a tough for the parents to provide guidance if there is no projected data on future earnings. What is the collateral anyways on a student loan? The fact you can’t claim bankruptcy and a right to a persons future earnings throughout their life? That isn’t right either. Loans should be made based on some type of projected output and if the loan obligations can’t be met over a certain period of time be treated like any other form of bankruptcy.
“if it can be rated it will be rated” – Rich Bartoncolleges definitely fall in that camp
It hardly seems like the feds could do a worse job than the existing rankings. My impression is that the most popular are focused on the attributes of the incoming class, how much money the program costs, and not on outcomes.I have significant reservations about your proposed system. My experience of school, having taken classes at 7 higher ed institutions and degrees from 2, is that I would not for an instance take my peers opinions on their experience as a guide for anything. Especially right after they graduated.My first supporting example is my graduation ceremony (I’m not going to say when) at the University of Michigan. The committee charged with picking a student speaker couldn’t decide between two finalists, so they both spoke. The first one looked like a newscaster (with a prominent jaw) and strung together every cliche you could possibly imagine. The class listened to that speech silently. The second one was a short, skinny Jewish guy who gave an excellent speech. As soon as he took the podium, my entire graduating class turned to each other, started talking, and didn’t listen to a word the guy said.When I went to business school, there was the required statistics class that everyone took and an optional statistics course that was prerequisite for some upper level classes. Both course covered the same material. The difference was that the optional course had a very demanding professor (not really worse than my high school football coach). After having taken both, I came to the conclusion that everyone could and should have taken the more demanding course (they certainly would have learned more and thus got more value), but that the administration was unwilling to listen to students complain or tolerate the bad feedback students might issue publicly. The result was that students got less real value out of their education.Finally I think you should take a look at this survey from Gallup that provides good insight into the factors in higher education that drive success and happiness later in life. Many seem like they offer aspects of college that could be measured by a rating system.
The fastest way to reform education:bring back iq and other tests so degrees can’t be used as a proxy. Make businesses own up to on the job training (tax breaks?) so that the state and students aren’t the ones responsible for trying to guess through college choice and major choice what businesses want.
Assessing colleges’ value is a complicated question, which is why more data is needed — both quantitive and qualitative. Open APIs will attract users, including the government, so we should stop arguing that they shouldn’t (do you want the recipient of your tax dollars to not analyze that data?). Just keep the data open to all. If this data is collected, there should be an effort to normalize for the demographic and other effects not related to the college itself. I want to know what the college does for each type of student so we can understand the why and how of their successes and failures.
What are the metrics by which these ratings would be compiled? Isn’t the efficacy of a college/university relative to the growth exhibited by the student while in attendance rather than an absolute measure of long term earning potential? Also, how do you strip out the myriad other factors (hard work, perseverance, dumb luck, etc) of success that have nothing to do with the school you attended?
Let’s see:(1) The ‘planets’ were the ‘wanderers’ because in the night sky they appeared to ‘wander’ in looping paths with respect to the fixed background of stars. Why? At least back to Ptolemy, people struggled. Copernicus, Kepler, etc. struggled. Finally Newton invented the calculus, the law of gravity, and his second law of motion, applied them to the motion of the planets, and made one of the greatest steps forward in the ascent of man. Now we know in rock solid terms about gravity and force equals mass times acceleration. We know about conservation of momentum and energy. Tap lightly with what Newton did and get the Navier-Stokes equations of fluid flow. A huge fraction of engineering stands solidly on what Newton did.(2) Run white light through a prism, and get out colors. Run each of those colors through another prism, and still get just that color. Take the colors and combine them and get back white light. So, white light is made of separate colors, and the colors cannot be decomposed further. Newton did that. Nice.(3) We learned about electricity and magnetism, and Maxwell and his equations made it all quite clear. Nearly everything in our use of electricity and electronics stands squarely on Maxwell’s equations.(4) Stare at Maxwell’s equations, see that motion of the laboratory is not in the equations, and start to conclude special relativity. From that tap lightly and get E = mc^2. Einstein did those. And the Michelson-Morley experiment added confirmation. Good work.But wait, there’s more: There is general relativity (e.g., as needed by GPS and much more), the microbe theory of disease, antibiotics, chemistry, chemical engineering, petrochemical engineering, pharmaceuticals, quantum mechanics, lasers, error correcting codes, public key cryptosystems, and much more. Oh, I don’t want to leave out the Hubble telescope, the 3 K background radiation and the big bang 13.7 billion years ago, little things like those.We’re making progress with DNA and cell biology and curing cancer and many other diseases. Moore’s law lets us have fantastic microprocessors sometimes for less than $100 each — a couple of families can blow that at lunch at McDonald’s.It does appear that such things have been by far the biggest contributions to the ascent of man and the work with by far the highest ROI of all.If want to understand such results and work, apply them, and add to them, then start as a freshman at a good college.Is that the best path to a high standard of living in Long Island? It was for James Simons but maybe not for everyone on Long Island.Is college the best path to a $800,000 house, three high end, new cars, a 50′ boat, etc.? No: Better is luck, to step into a cushy slot in a good family business, to make a big splash as a hip hop musician, NBA, or NFL athlete.But such paths have little promise of building a quantum computer, curing cancer, explaining dark energy or dark matter, calculating the spacial power spectra of the 3 K background radiation and using it to test the big bang theory, etc. But it does appear that our society does want to pursue such research directions; so, we have the NSF and NIH. Again, for people who want to pursue such things, start as a freshman at a good college.A good college is one thing; a trade school is something else. We also have a lot of trade schools and junior colleges. It does appear, however, that just for the goal of an $800,000 house, a good college is a better path than a trade school.Yes, SnapChat was a big success, but it’s already been done now. To generalize to another social, local, mobile app with some proprietary data handling to get teenage girls to squeal and be imprudent is not necessarily the way to another billion dollar company.When I was scheduling the fleet at FedEx, I got a good first solution quickly with just some routine software, the law of cosines for spherical triangles, etc.. But much more was quite important, and that took me straight into work right up against the question of P versus NP and some of the latest research in pure and applied math. Trade school is not a good path to getting there.For ‘evaluating’ the best US research universities, that is done everyday much like evaluating the best teams in the NFL or NBA: That research work is very competitive. For what college work is good preparation for such research, that is also quite well known. E.g., for math it’s calculus, linear algebra, abstract algebra, advanced calculus, differential equations, topics in applied advanced calculus, elementary probability and statistics, measure theory and functional analysis, differential geometry, calculus on manifolds, etc. Can do much the same for physics, etc. And good high school work, e,g,, good SAT scores, as preparation for the college work is also quite well known.Net: (1) It’s not so clear just what to change about education at the best, e.g., Harvard, Princeton, Berkeley, US colleges, universities, and graduate and professional schools, for individuals or our society. I do not believe that the head of HR at Google or Peter Thiel know. (2) It is not so clear what to do to build the next billion dollar information technology company or what education would be a big help to that end.
Because grading high schools worked so well? Same argument, disastrous results.
One often measures a person by what he or she has attained in life at some static point. But a more revealing and accurate assessment is probably one based upon where this individual started off in life, compared to where they are when assessed. The progress made should be what the rating is based upon. I feel that the same is true of colleges and universities. My first university catered to low-income, minority students who were lucky to attend one year of college at all, much less graduate from one in less than 5 years. The question I would ask is how do you compare that school against the one down the street which on paper offered the same range of curricula but to students from a higher economic category. Their statistics beat us hands down, their students got into less trouble, and statistics on recruitment into corporate positions cannot even be compared, but were they a “better” school? Just my two cents worth.
Nobody can check this site except (MOD) http://goo.gl/HxErof
Why not grade colleges? I mean, the government grades financial investments and what not and look how well that has protected us against major financial catastrophes.
Kinda sounds like a Glassdoor for colleges
College is what you make of it.
this slide is epic.
One metric that would greatly increase accountability wouldbe student learning outcomes. Scholars have developed good measures of studentlearning in college only in the last decade; when they were applieda few years ago, readers were shocked at how many students learn little ornothing in college. More than a third showed no statistically significant cognitive gains. (Unsurprisingly, business leaders have hailed thedevelopment of reliable measures for core skills).If the federal government tied access to Title IV money tothose schools that assessed student learning and published their results, theywould help consumers seek out (and therefore reward) effective colleges muchmore efficiently.
It depends whether learning outcomes are relevant or not. Why are we providing subsidies for kids to go to college? If the answer is a shot at a higher income, then we should measure that directly. There is serious debate in the economics community as to whether you get ahigher income after college because of the training you received.
Sites like @Upstart and @SofFi give us valuable data on student outcomes already. How can we leverage this? It seems like we rank colleges already, so a better, more transparent system would be to aggregate government data sets in open sourced communities in order to capture student outcomes.
Some type of opt-in yearly newsletter with a simple survey that asks all graduates regardless of school they went to questions similar to these would be interesting to gather insights from over time:1) Are you currently working in a profession related to your most recently acquired degree?2) On a scale of 1-10, how useful did you find your most recent post-secondary education as it relates to your current profession?3) What school did you go to last?4) What program did you take? / What degree do you have?5) Have you ever recommended this program/degree to someone else?6) Do you enjoy doing the work you do today?7) Are you planning on changing careers in the near future? If yes, why?Question laddering techniques could be used in the survey based on what answers are given by respondents to dig deeper into potential problem areas when they appear rather than overwhelm people with a million questions to answer all at once.LinkedIn could probably do this very effectively every year by gathering insights from everyone who has declared they have a post-secondary education via a yearly celebrated email blast. They could make it a yearly movement to help increase transparency on education quality everywhere in the world. They could analyze and publish a report online for free every year and then sell the detailed data back to each university (or give it back to them for free!) so that the schools can learn about which programs are doing well, which aren’t, and to get a general sense of how they stack up to schools that are closely competitive to them and that are industry leaders in certain areas. In addition to the survey data, LinkedIn would also be able to cross reference how many jobs you’ve had after the degree was acquired and how long, among other interesting information (how many views their profile gets, number of connections on average, etc. etc.)
Why do we subsidize college? Until we can answer that question, all measures are pointless.