Blueshift
  • ENGAGE21 – LIVE!
  • Product  
    •  PLATFORM
      • What is a SmartHub CDP?
      • Omnichannel Orchestration
      • Predictive Intelligence
      • Single Customer View
      • Audience Segmentation
      • 1:1 Personalization
    •  SOLUTIONS
      • Email Automation
      • Mobile Marketing
      • Website Personalization
      • Audience Targeting
      • Contextual Chat
    •  SUPPORT
      • Technology Integrations
      • Product Updates
      • Documentation
      • Support Plans
      • Pricing
  • Customers
  • Resources  
    •   Smart Guides
    •   Case Studies
    •   Videos
    •   Reports
    •   Events
    •   Academy
    •   Blog
  • Company  
    •   About Blueshift
    •   News & Awards
    •   Careers
  • Contact Us
  • LOGIN
  • Search
  • Menu Menu
  • Product
    •  PLATFORM
      • SmartHub CDP
      • Omnichannel Orchestration
      • Predictive Intelligence
      • Single Customer View
      • Audience Segmentation
      • 1:1 Personalization
    •  SOLUTIONS
      • Email Automation
      • Mobile Marketing
      • Website Personalization
      • Audience Targeting
      • Contextual Chat
    •  SUPPORT
      • Technology Integrations
      • Product Updates
      • Documentation
      • Support Plans
      • Pricing
  • Customers
  • Resources
    •   Smart Guides
    •   Case Studies
    •   Videos
    •   Reports
    •   Events
    •   Blog
    •   Academy
  • Company
    •   About Blueshift
    •   News & Awards
    •   Careers
  • ENGAGE21
  • LOGIN
  • Contact Us

How do you know if your model is going to work? Part 1: The problem

Digital Marketing Practices & Insights
Model Data

This month we have a guest post series from our dear friend and advisor, John Mount, on building reliable predictive models. We are honored to share his hard won learnings with the world.

Authors: John Mount (more articles) and Nina Zumel (more articles) of Win-Vector LLC.

“Essentially, all models are wrong, but some are useful.”
George Box

Here’s a caricature of a data science project: your company or client needs information (usually to make a decision). Your job is to build a model to predict that information. You fit a model, perhaps several, to available data and evaluate them to find the best. Then you cross your fingers that your chosen model doesn’t crash and burn in the real world.

We’ve discussed detecting if your data has a signal. Now: how do you know that your model is good? And how sure are you that it’s better than the models that you rejected?

Bartolomeu Velho 1568
Geocentric illustration Bartolomeu Velho, 1568 (Bibliothèque Nationale, Paris)

 

Notice the Sun in the 4th revolution about the earth. A very pretty, but not entirely reliable model.

In this latest “Statistics as it should be” series, we will systematically look at what to worry about and what to check. This is standard material, but presented in a “data science” oriented manner. Meaning we are going to consider scoring system utility in terms of service to a negotiable business goal (one of the many ways data science differs from pure machine learning).

To organize the ideas into digestible chunks, we are presenting this article as a four part series (to finished in the next 3 Tuesdays). This part (part 1) sets up the specific problem.

Read more.

September 3, 2015/by Alan Brusky
Tags: Metrics
Share this entry
  • Share on Facebook
  • Share on Twitter
  • Share on WhatsApp
  • Share on Pinterest
  • Share on LinkedIn
  • Share on Reddit
  • Share by Mail
https://blueshift.com/wp-content/uploads/How-do-you-know-if-your-model-Part-1-V2.jpg 640 1200 Alan Brusky https://blueshift.com/wp-content/uploads/blueshift-primary.svg Alan Brusky2015-09-03 21:36:042019-11-22 10:56:18How do you know if your model is going to work? Part 1: The problem

Recent Articles

  • Customer data fuels the online experience - personalization increases site engagement by over 55%. Why Customer Data is Essential to the Online ExperienceApril 21, 2021 - 6:54 am
  • Conference Recap: How to Create Continuous Customer EngagementApril 16, 2021 - 8:02 am
  • Learn to Truly Leverage Your Data in Our Smart Guide to Data ActivationApril 14, 2021 - 6:39 am
  • Man receives product recommendations on smartphone, intended to boost engagement. Boosting Engagement Throughout the Customer Experience with RecommendationsApril 8, 2021 - 8:37 am
  • CDPs vs. Marketing Automation Platforms: What Marketers Need to KnowApril 7, 2021 - 6:18 am
  • Welcoming New Customers: How to Leverage Recommendations in Retail and EcommerceApril 1, 2021 - 6:31 am

Headquarters

433 California St, Suite 600,
San Francisco, CA 94104

Global Offices

Charlotte, NC
Pune, India
London, UK

hello@blueshift.com

Company

  • About Blueshift
  • Customers
  • Careers
  • News and Awards
  • Agency Partners
  • Contact Us

Platform

  • SmartHub CDP
  • Single Customer View
  • Audience Segmentation
  • Predictive Intelligence
  • 1:1 Personalization
  • Omnichannel Orchestration
  • Technology Integrations

Solutions

  • Email Automation
  • Mobile Marketing
  • Website Personalization
  • Audience Targeting
  • Contextual Chat

Resources

  • Smart Guides
  • Case Studies
  • Blueshift Academy
  • Product Updates
  • Documentation
  • Developer Portal
  • RFP Guide

© 2021 COPYRIGHT BLUESHIFT LABS, INC. PRIVACY POLICY   |   TERMS OF SERVICE   |   ANTI-SPAM POLICY

Scroll to top