Blueshift
  • Product  
    • PLATFORM
      • SmartHub CDP
      • Omnichannel Orchestration
      • Predictive Intelligence
      • Single Customer View
      • Audience Segmentation
      • 1:1 Personalization
    • SOLUTIONS
      • Email Automation
      • Mobile Marketing
      • Website Personalization
      • Audience Targeting
      • Contextual Chat
    • PLANS AND INTEGRATIONS
      • Integration Partners
      • Support Plans
      • Pricing
  • Customers
  • Resources  
    • Library
    • Blog
    • Videos
    • Documentation
    • Product Updates
    • Blueshift Academy
  • Company  
    • About Blueshift
    • Events
    • News & Awards
    • Careers
  • Contact Us
  • LOGIN
  • Search
  • Menu Menu
  • Product
    • PLATFORM
      • SmartHub CDP
      • Omnichannel Orchestration
      • Predictive Intelligence
      • Single Customer View
      • Audience Segmentation
      • 1:1 Personalization
    • SOLUTIONS
      • Email Automation
      • Mobile Marketing
      • Website Personalization
      • Contextual Chat
      • Retired – Audience Targeting
    • PLANS AND INTEGRATIONS
      • Integration Partners
      • Support Plans
      • Pricing
  • Customers
  • Resources
    • Library
    • Blog
    • Videos
    • Documentation
    • Product Updates
    • Blueshift Academy
  • Company
    • About Blueshift
    • Events
    • News & Awards
    • Careers
  • LOGIN
  • Contact Us

How do you know if your model is going to work? Part 1: The problem

Digital Marketing Practices & Insights
Model Data

This month we have a guest post series from our dear friend and advisor, John Mount, on building reliable predictive models. We are honored to share his hard won learnings with the world.

Authors: John Mount (more articles) and Nina Zumel (more articles) of Win-Vector LLC.

“Essentially, all models are wrong, but some are useful.”
George Box

Here’s a caricature of a data science project: your company or client needs information (usually to make a decision). Your job is to build a model to predict that information. You fit a model, perhaps several, to available data and evaluate them to find the best. Then you cross your fingers that your chosen model doesn’t crash and burn in the real world.

We’ve discussed detecting if your data has a signal. Now: how do you know that your model is good? And how sure are you that it’s better than the models that you rejected?

Bartolomeu Velho 1568
Geocentric illustration Bartolomeu Velho, 1568 (Bibliothèque Nationale, Paris)

 

Notice the Sun in the 4th revolution about the earth. A very pretty, but not entirely reliable model.

In this latest “Statistics as it should be” series, we will systematically look at what to worry about and what to check. This is standard material, but presented in a “data science” oriented manner. Meaning we are going to consider scoring system utility in terms of service to a negotiable business goal (one of the many ways data science differs from pure machine learning).

To organize the ideas into digestible chunks, we are presenting this article as a four part series (to finished in the next 3 Tuesdays). This part (part 1) sets up the specific problem.

Read more.

September 3, 2015/by Alan Brusky
Tags: Metrics
Share this entry
  • Share on Facebook
  • Share on Twitter
  • Share on WhatsApp
  • Share on Pinterest
  • Share on LinkedIn
  • Share on Reddit
  • Share by Mail
https://blueshift.com/wp-content/uploads/How-do-you-know-if-your-model-Part-1-V2.jpg 640 1200 Alan Brusky https://blueshift.com/wp-content/uploads/blueshift-primary.svg Alan Brusky2015-09-03 21:36:042019-11-22 10:56:18How do you know if your model is going to work? Part 1: The problem

Recent Articles

  • Customer Data Platform experts David Raab and Shamir Duverseau join Blueshift in our exclusive webinar. Webinar Recap: What the CDP?! With the CDP Institute and Smart Panda LabsJanuary 15, 2021 - 5:10 am
  • new retail cx and marketing trends 8 Retail CX and Marketing Trends for the New YearJanuary 13, 2021 - 5:19 am
  • ai recommendations saw an 81% increase in marketing revenue Why You Need AI-Powered Recommendations in Your MarketingJanuary 7, 2021 - 6:07 am
  • hackathon 2020 winners Blueshift’s 2020 Hackathon RecapDecember 18, 2020 - 6:45 am
  • Creating Dynamic Templates for Mobile Channels is Easier than Ever with Blueshift’s Enhanced Creative StudioDecember 16, 2020 - 1:51 am
  • G2 Momentum Leader in Marketing Automation and CDP Grids Winter 2021 Blueshift Named a Momentum Leader for CDP and Marketing Automation Categories in G2’s Winter 2021 ReportDecember 15, 2020 - 6:25 am

Headquarters

433 California St, Suite 600,
San Francisco, CA 94104

Global Offices

Charlotte, NC
Pune, India
London, UK

hello@blueshift.com

Company

  • About Blueshift
  • Customers
  • News and Awards
  • Events
  • Careers
  • Contact Us

Platform

  • SmartHub CDP
  • Single Customer View
  • Audience Segmentation
  • Predictive Intelligence
  • 1:1 Personalization
  • Omnichannel Orchestration
  • Integration Partners

Solutions

  • Email Automation
  • Mobile Marketing
  • Website Personalization
  • Audience Targeting
  • Contextual Chat

Resources

  • Documentation
  • Developer Portal
  • Product Updates
  • Case Studies
  • Reports
  • RFP Guide
  • Blueshift Academy

© 2020 COPYRIGHT BLUESHIFT LABS, INC. PRIVACY POLICY   |   TERMS OF SERVICE   |   ANTI-SPAM POLICY

Scroll to top