The Age of Big Data

 GOOD with numbers? Fascinated by data? The sound you hear is opportunity knocking.

Mo Zhou was snapped up by I.B.M. last summer, as a freshly minted Yale M.B.A., to join the technology company’s fast-growing ranks of data consultants. They help businesses make sense of an explosion of data — Web traffic and social network comments, as well as software and sensors that monitor shipments, suppliers and customers — to guide decisions, trim costs and lift sales. “I’ve always had a love of numbers,” says Ms. Zhou, whose job as a data analyst suits her skills.


To exploit the data flood, America will need many more like her. A report last year by the McKinsey Global Institute, the research arm of the consulting firm, projected that the United States needs 140,000 to 190,000 more workers with “deep analytical” expertise and 1.5 million more data-literate managers, whether retrained or hired.




In business, economics and other fields, Professor Brynjolfsson says, decisions will increasingly be based on data and analysis rather than on experience and intuition. “We can start being a lot more scientific,” he observes.

There is plenty of anecdotal evidence of the payoff from data-first thinking. The best-known is still “Moneyball,” the 2003 book by Michael Lewis, chronicling how the low-budget Oakland A’s massaged data and arcane baseball statistics to spot undervalued players. Heavy data analysis had become standard not only in baseball but also in other sports, including English soccer, well before last year’s movie version of “Moneyball,” starring Brad Pitt.

Retailers, like Walmart and Kohl’s, analyze sales, pricing and economic, demographic and weather data to tailor product selections at particular stores and determine the timing of price markdowns. Shipping companies, like U.P.S., mine data on truck delivery times and traffic patterns to fine-tune routing.

Online dating services, like Match.com, constantly sift through their Web listings of personal characteristics, reactions and communications to improve the algorithms for matching men and women on dates. Police departments across the country, led by New York’s, use computerized mapping and analysis of variables like historical arrest patterns, paydays, sporting events, rainfall and holidays to try to predict likely crime “hot spots” and deploy officers there in advance.


Big Data is already transforming the study of how social networks function. In the 1960s, Stanley Milgram of Harvard used packages as his research medium in a famous experiment in social connections. He sent packages to volunteers in the Midwest, instructing them to get the packages to strangers in Boston, but not directly; participants could mail a package only to someone they knew. The average number of times a package changed hands was remarkably few, about six. It was a classic demonstration of the “small-world phenomenon,” captured in the popular phrase “six degrees of separation.”

_______________________________________________________________________________

More info:

website link: https://databasescientist.org/

contact us:  contact@databasescientist.org

Nomination Link: https://databasescientist.org/award-nomination/?ecategory=Awards&rcategory=Awardee

_______________________________________________________________________________

social media:

Twitter: https://x.com/databasesc10061

Pinterest: https://in.pinterest.com/databasescientist/

Linked in: https://www.linkedin.com/in/databasescientist-database-440a12365/

Instagram link: https://www.instagram.com/databasescientist/

YouTube: https://www.youtube.com/@databasescientist






Ad


Comments

Post a Comment

Popular posts from this blog

Large Language Models and Vector Databases for News Recommendations

Image
  Large Language Models   (LLMs) generated a global buzz in the machine learning community with recent releases of generative AI tools such as Chat-GPT, Bard, and others alike. One of the core ideas behind these solutions is to compute a numerical representation of unstructured data (such as texts and images) and find similarities between these representations. However, taking all of these concepts into a production environment has its own set of machine learning engineering challenges: How to generate these representations quickly? How to store them in a proper database? How to quickly compute similarities for production environments? In this article, I introduce two open-source solutions that aim to solve these questions: Sentence Transformers  [1] : an embedding generation technique based on textual information and; Qdrant : a vector database capable of storing embeddings and providing an easy interface to query them. These tools are applied to NPR [2], a News Portal R...
Read more

Is Palantir creating a national database of US citizens?

Image
According to its website, Palantir makes "products for human-driven analysis of real-world data." On May 30, 2025,  The New York Times published an article   titled "Trump Taps Palantir to Compile Data on Americans," detailing a supposed combined effort between the U.S. federal government and the data software company Palantir to centralize data on American citizens. Data privacy advocates did not take the news well, calling it "dystopian" and a massive invasion of privacy. Four days later, Palantir posted a statement to X responding to the article, calling the reporting " blatantly untrue" because "Palantir never collects data to unlawfully surveil Americans, and our Foundry platform employs granular security protections." On June 9, 2025, the company followed up with a long blog post on X titled " Correcting the Record: Responses to the May 30, 2025 New York Times Article on Palantir," which aimed to fact-check the article....
Read more

NIH autism database announcement raises concerns among researchers

Image
A new research initiative from the U.S. National Institutes of Health (NIH) seeks to gather and analyze multiple autism-related datasets to better understand the underlying causes of the condition. Letters of intent are due tomorrow, but the project’s atypical call for applications has autism scientists divided on whether or not to participate. The Autism Data Science Initiative provides $50 million for an anticipated 10 to 25 projects, according to the 27 May announcement. The funding opportunity uses an Other Transaction (OT) funding mechanism, which tends to be implemented when, among other reasons, a project is expected to evolve over time, more collaboration is required from the NIH or nontraditional research groups are involved, according to an NIH website. For a field that has recently seen budget cuts, some researchers are welcoming the opportunity for funding, says   Helen Tager-Flusberg , director of the Center for Autism Research Excellence at Bosto...
Read more