Data Skeptic (general)

In this episode, the data scientist Wentao Su shares his experience in AB testing on social media platforms like LinkedIn and TikTok.

We talk about how network science can enhance AB testing by accounting for complex social interactions, especially in environments where users are both viewers and content creators. These interactions might cause a "spillover effect" meaning a possible influence across experimental groups, which can distort results.

To mitigate this effect, our guest presents heuristics and algorithms they developed ("one-degree label propagation”) to allow for good results on big data with minimal running time and so optimize user experience and advertiser performance in social media platforms.

Direct download: 558-networks-for-ab-testing.mp3
Category:general -- posted at: 7:00am PST

Alex Bisberg, a PhD candidate at the University of Southern California, specializes in network science and game analytics, with a focus on understanding social and competitive success in multiplayer online games.

In this episode, listeners can expect to learn from a network perspective about players interactions and patterns of behavior. Through his research on games, Alex sheds light on how network analysis and statistical tests might explain positive contagious behaviors, such as generosity, and explore the dynamics of collaboration and competition in gaming environments. These insights offer valuable lessons not only for game developers in enhancing player experience, engagement and retention, but also for anyone interested in understanding the ways that virtual interactions shape social networks and behavior.

Direct download: lessons-from-egamer-networks.mp3
Category:general -- posted at: 7:00am PST

In this episode we discuss the GitHub Collaboration Network with Behnaz Moradi-Jamei, assistant professor at James Madison University.  As a network scientist, Behnaz created and analyzed a network of about 700,000 contributors to Github's repository.  The network of collaborators on GitHub was created by identifying developers (nodes) and linking them with edges based on shared contributions to the same repositories. This means that if two developers contributed to the same project, an edge (connection) was formed between them, representing a collaborative relationship network consisting of 32 million such connections.
By using algorithms for Community Detection, Behnaz's analysis reveals insights into how developer communities form, function, and evolve, that can be used as guidance for OSS community managers.

Direct download: github-collaboration-network-fixed.mp3
Category:general -- posted at: 12:49pm PST

We are joined by Abhishek Paudel, a PhD Student at George Mason University with a research focus on robotics, machine learning, and planning under uncertainty, using graph-based methods to enhance robot behavior. He explains how graph-based approaches can model environments, capture spatial relationships, and provide a framework for integrating multiple levels of planning and decision-making.

Direct download: graphs-and-ml-for-robotics.mp3
Category:general -- posted at: 6:00am PST

We are joined by Maciej Besta, a senior researcher of sparse graph computations and large language models at the Scalable Parallel Computing Lab (SPCL). In this episode, we explore the intersection of graph theory and high-performance computing (HPC), Graph Neural Networks (GNNs) and LLMs.

Direct download: graphs-for-hpc-and-llms.mp3
Category:general -- posted at: 8:13am PST

In this episode, we sit down with Yuanyuan Tian, a principal scientist manager at Microsoft Gray Systems Lab, to discuss the evolving role of graph databases in various industries such as fraud detection in finance and insurance, security, healthcare, and supply chain optimization. 

Direct download: graph-database-and-ai.mp3
Category:general -- posted at: 6:24am PST

Our new season "Graphs and Networks" begins here!  We are joined by new co-host Asaf Shapira, a network analysis consultant and the podcaster of NETfrix – the network science podcast. Kyle and Asaf discuss ideas to cover in the season and explore Asaf's work in the field.

Direct download: network-analysis-in-practice.mp3
Category:general -- posted at: 7:45am PST

Join us for our capstone episode on the Animal Intelligence season. We recap what we loved, what we learned, and things we wish we had gotten to spend more time on. This is a great episode to see how the podcast is produced. Now that the season is ending, our current co-host, Becky, is moving to emeritus status. In this last installment we got to spend a little more time getting to know Becky and where her work will take her after this. Did Data Skeptic inspire her to learn more about machine learning? Tune in and find out. 

Direct download: animal-intelligence-final-exam.mp3
Category:general -- posted at: 9:16am PST

David Obembe, a recent University of Tartu graduate, discussed his Masters thesis on integrating LLMs with process mining tools. He explained how process mining uses event logs to create maps that identify inefficiencies in business processes. David shared his research on LLMs' potential to enhance process mining, including experiments evaluating their performance and future improvements using Retrieval Augmented Generation (RAG).

Direct download: process-mining-with-llms.mp3
Category:general -- posted at: 8:08am PST

Our guest today is Risa Shinoda, a PhD student at Kyoto University Agricultural Systems Engineering Lab, where she applies computer vision techniques.

She talked about the OpenAnimalTracks dataset and what it was used for. The dataset helps researchers predict animal footprint. She also discussed how she built a model for predicting tracks of animals. She shared the algorithms used and the accuracy they achieved. She also discussed further improvement opportunities for the model.

Direct download: open-animal-tracks.mp3
Category:general -- posted at: 9:58am PST

This episode features an interview with Mélisande Teng, a PhD candidate at Université de Montréal. Her research lies in the intersection of remote sensing and computer vision for biodiversity monitoring.

Direct download: bird-distribution-modeling-with-satbird.mp3
Category:general -- posted at: 1:58pm PST

In this interview with author Deborah Gordon, Kyle asks questions about the mechanisms at work in an ant colony and what ants might teach us about how to build artificial intelligence. Ants are surprisingly adaptive creatures whose behavior emerges from their complex interactions. Aspects of network theory and the statistical nature of ant behavior are just some of the interesting details you'll get in this episode.

Direct download: ant-encounters.mp3
Category:general -- posted at: 7:01am PST

This season it’s become clear that computing skills are vital for working in the natural sciences. In this episode, we were fortunate to speak with Madlen Wilmes, co-author of the book "Computing Skills for Biologists: A Toolbox". We discussed the book and why it’s a great resource for students and teachers. In addition to the book, Madlen shared her experience and advice on transitioning from academia to an industry career and how data analytic skills transfer to jobs that your professionals might not always consider. Join us and learn more about the book and careers using transferable skills.

Direct download: computing-toolbox.mp3
Category:general -- posted at: 7:25am PST

In this episode, we talked shop with Hager Radi about her biodiversity monitoring work. While biodiversity modeling may sound simple, count organisms and mark their location, there is a lot more to it than that! Incomplete and biased data can make estimations hard. There are also many species with very few observations in the wild. Using machine learning and remote sensing data, scientists can build models that predict species distributions with limited data. Listen in and hear about Hager’s work tackling these challenges and the tools she has built.

Direct download: biodiversity-monitoring.mp3
Category:general -- posted at: 7:04pm PST

Today, Ashay Aswale and Tony Lopez shared their work on swarm robotics and what they have learned from ants. Robotic swarms must solve the same problems that eusocial insects do. What if your pheromone trail goes cold? What if you’re getting bad information from a bad-actor within the swarm? Answering these questions can help tackle serious robotic challenges. For example, a swarm of robots can lose a few members to accidents and malfunctions, but a large robot cannot. Additionally, a swarm could be host to many castes like an ant colony. Specialization with redundancy built in seems like a win-win! Tune in and hear more about this fascinating topic.

Direct download: hacking-the-colony.mp3
Category:general -- posted at: 8:13am PST

During this season we have talked with researchers working to utilize machine learning for behavioral observations. In previous episodes, you have heard about the software people like Richard use, but you haven’t heard much from scientists modifying and using these tools for specific research cases. PhD student, Richard Vogg, is working with multi-camera set-ups to track lemurs and macaques solving puzzle boxes in the wild. His work is part of a larger movement to automate behavioral analyses of video data. Listen in and learn why this tech is useful and why multi-camera setups are a good idea for more reliably identifying poses and individual animals.

Direct download: primate-poses.mp3
Category:general -- posted at: 1:41pm PST

Generative AI can struggle to create realistic animals and 2D representations often have mistakes like extra limbs and tails. If 2D wasn’t hard enough, there are researchers working on generative 3D models. 3D models present an extra challenge because there is paucity of training datasets.In this episode, PhD students Sandeep and Oindrila walked us through their work on creating 3D animals using 2D data. Join us to learn about their pipelines, quality control, tie in with iNaturalist, and how this tech could streamline FX pipelines.

Direct download: generating-3d-animals.mp3
Category:general -- posted at: 9:17am PST

Today, we sat down with Dr. Ignacio Escalante Meza to learn about opiliones and treehoppers. Opiliones, known as “daddy long legs” in the US, are understudied arachnids known for their tenacious locomotor behavior, sociality, and chemical communication. Treehoppers communicate through the stems of plants using vibrations. They can signal danger, attract mates, and communicate with their offspring. Join us to learn how researchers turn their vibrations into sound waves and study what they have to say.

Direct download: weird-communication.mp3
Category:general -- posted at: 6:00am PST

Human shipping operations have increased significantly in the past few decades.  While that means international trade and cheap goods for humans, it also means the ocean has experienced an increase in noise pollution.  This has a measurable negative impact on marine mammals and other aquatic life.  Could mathematics be the solution?  This interview explores how optimization techniques can guide voyage optimization in a way that handles multiple optimization objectives including fuel cost and sound reduction.

Direct download: reducing-the-impact-of-ship-noise-on-marine-mammals.mp3
Category:general -- posted at: 5:00am PST

Robbie Moon from the Georgia Tech Scheller College of Business joins us to discuss the analysis of unstructured data and the application of NLP methodologies towards financial data.

Direct download: analysis-of-unstructured-data.mp3
Category:general -- posted at: 11:50am PST

Have you ever participated in citizen science? Do you want to? One of the most popular platforms for crowdsourcing biodiversity data is iNaturalist. In addition to being a great science tool, the iNaturalist app can help you identify the organisms you encounter every day. We talked to Executive Director Scott Laurie about how scientists use iNaturalist. We also got to discuss what makes iNaturalist’s AI species recognition so good, and how citizen scientists are constantly providing high-quality training data. Listen in and learn how this fun-to-use tool works, where it's headed, and how you can get involved.

Direct download: inaturalist.mp3
Category:general -- posted at: 1:58pm PST

Do you code or are you interested in learning to code? Join us today and hear from three individuals that are at very different stages of their coding journeys. Becky Hansis-O’Neill (also our co-host this season) shares her experiences as a newbie who wants to learn more. Dr. Malia Gehan, a self-taught developer interested in studying plant phenotypes, explains why and how she and her colleagues learned to code and developed PlantCV. Finally, Dr. John Wilmes discusses his work as a professional mathematician and Machine Learning Research Engineer. Whether you are thinking about learning to code or an expert, we’re sure you will see a bit of yourself in this episode. 

Direct download: learn-to-code.mp3
Category:general -- posted at: 1:10pm PST

You’ve heard of Human Computer Interaction (HCI), now get ready for Animal Computer Interaction (ACI). Ilyena has made a career developing computer interfaces for non-human animals. She has worked with dogs, parrots, primates, and even giraffes. This is challenging because animals have a wide range of abilities and preferences. Parrots, for example, use their tongues to make selections on touchscreens. Listen in on our conversation and learn about interface development and testing with animals and how technology may improve animal welfare. 

Direct download: animal-computer-interaction.mp3
Category:general -- posted at: 6:19am PST

Cat observes great apes in the wild and in the lab to crack the code of their gestural communication. We discussed the challenges and benefits of studying apes in the wild vs in the lab. Cat also shared how her lab identifies and studies ape gestures. It turns out that humans are pretty good at guessing what apes are trying to communicate with one another. Join us in this episode to learn more about the evolution of communication in great apes, and what we can learn from our closest relatives. 

Direct download: ape-gestures.mp3
Category:general -- posted at: 6:00am PST

In this episode, Kozzy discusses his endeavors to compare the cognitive abilities of humans, animals, and AI programs. Specifically, we discussed object permanence, the ability to understand an object still exists in space even when you can’t see it. Our conversation traverses both philosophical and practical questions surrounding AI evaluation. We also learned about Animal AI 3, a gaming environment developed in Unity where AI programs and humans can go head-to-head to solve different problems in a gaming environment.

Direct download: evaluating-ai-abilities.mp3
Category:general -- posted at: 9:06am PST

Théo Michelot has made a career out of tackling tough ecological questions using time-series data. How do scientists turn a series of GPS location observations over time into useful behavioral data? GPS tech has improved to the point that modern data sets are large and complex. In this episode, Théo takes us through his research and the application of Hidden Markov Models to complex time series data. If you have ever wondered what biologists do with data from those GPS collars you have seen on TV, this is the episode for you! 

Direct download: hmm-for-behavior.mp3
Category:general -- posted at: 6:00am PST

Brian Taylor shares his research on magnetoreception. Animals like birds and sea turtles use magnetoreception to use the Earth’s magnetic field for navigation, but it’s not a sense that’s well understood. Brian uses animal magnetoreception to engineer new ways to navigate the globe. Even cooler, he also takes hypotheses for how magnetoreception works in animals and uses computational simulations to digitally test them. Check out this episode to hear more about Brian’s research and learn more about this little known sensory ability. 

Direct download: bioinspired-engineering.mp3
Category:general -- posted at: 8:31am PST

Modeling evolutionary processes goes way beyond the Hardy-Weinberg Equilibrium we all learned in biology class. Natural selection comes from many sources like resources availability, mate preferences, competition. Modeling entire populations of organisms of different species is the holy grail of digital evolution. Join our discussion with evolutionary biologist and software engineer Ben Haller to learn about his work on SLiM and how it helps other biologists model population genetics over time. 

Direct download: modelling-evolution.mp3
Category:general -- posted at: 7:06pm PST

It’s almost impossible to think about animal behavior without thinking of dogs! Our canine friends are a subspecies of wolf that has been co-evolving with us for tens of thousands of years. The transition from wolf to pet has required intense natural and artificial selection for behaviors that allow dogs to live alongside humans, but behavior is not so simple. Join us for a discussion with Dr. Jessica Hekman and learn about dog welfare, behavioral genetics, and the quest to understand the dogs in our lives. 

Direct download: behavioral-genetics.mp3
Category:general -- posted at: 6:57am PST

In this episode, we are joined by Barbara Webb and Anna Hadjitofi. Barbara runs the Insect Robotics lab at the University of Edinburgh, and Anna is a PhD student at the School of Informatics at the university. She is interested in studying and understanding the neural mechanism of the honeybee waggle dance. They join us to discuss the paper: Dynamic antennal positioning allows honeybee followers to decode the dance.

Direct download: signal-in-the-noise.mp3
Category:general -- posted at: 5:03pm PST

Many researchers and students have painstakingly labeled precise details about the body positions of the creatures they study. Can AI be used for this labeling? Of course it can! Today's episode discusses Social LEAP Estimates Animal Poses (SLEAP), a software solution to train AI to perform this tedious but important labeling work.

Direct download: pose-tracking.mp3
Category:general -- posted at: 11:59pm PST

Our guest in this episode is Sebastien Motsch, an assistant professor at Arizona State University, working in the School of Mathematical and Statistical Science. He works on modeling self-organized biological systems to understand how complex patterns emerge.

Direct download: modeling-group-behavior.mp3
Category:general -- posted at: 6:00am PST

Our guest in this episode is Ryan Hanscom. Ryan is a Ph.D. candidate in a joint doctoral evolution program at San Diego State University and the University of California, Riverside. He is a terrestrial ecologist with a focus on herpetology and mammalogy.  Ryan discussed how the behavior of rattlesnakes is studied in the natural world, particularly with an increase in temperature.

Direct download: advances-in-data-loggers.mp3
Category:general -- posted at: 6:04am PST

We are joined by Hank Schlinger, a professor of psychology at California State University, Los Angeles. His research revolves around theoretical issues in psychology and behavioral analysis.  Hank establishes that words have references and questions the reference for intelligence. He discussed how intelligence can be observed in animals. He also discussed how intelligence is measured in a given context.

Direct download: what-you-know-about-intelligence-is-wrong-fixed.mp3
Category:general -- posted at: 11:31pm PST

On today’s episode, we are joined by Aimee Dunlap. Aimee is an assistant professor at the University of Missouri–St. Louis and the interim director at the Whitney R. Harris World Ecology Center.

Aimee discussed how animals perceive information and what they use it for. She discussed the connection between their environment and learning for decision-making. She also discussed the costs required for learning and factors that affect animal learning.

Direct download: animal-decision-making.mp3
Category:general -- posted at: 2:55pm PST

We are joined by Tamar Gutnick, a visiting professor at the University of Naples Federico II, Napoli, Italy. She studies the octopus nervous system and their behavior, focusing on cognition and learning behaviors.

Tamar gave a background to the kind of research she does — lab research. She discussed some challenges with observing octopuses in the lab. She discussed some patterns observed by the octopus lifestyle in a controlled setting.

Tamar discussed what they know about octopus intelligence. She discussed the octopus nervous system and why they are unique compared to other animals. She discussed how they measure the behavior of octopuses using a video recording and a logger to track brain activity.

Direct download: octopus-cognition.mp3
Category:general -- posted at: 4:50pm PST

Claire Hemmingway, an assistant professor in the Department of Psychology and Ecology and Evolutionary Biology at the University of Tennessee in Knoxville, is our guest today. Her research is on decision-making in animal cognition, focusing on neotropical bats and bumblebees.

Claire discussed how bumblebees make foraging decisions and how they communicate when foraging. She discussed how they set up experiments in the lab to address questions about bumblebees foraging. She also discussed some nuances between bees in the lab and those in the wild.

Claire discussed factors that drive an animal's foraging decisions. She explained the foraging theory and how a colony works together to optimize its foraging. She also touched on some irrational foraging behaviors she observed in her study.

Claire discussed some techniques bees use to learn from past behaviors. She discussed the effect of climate change on foraging bees' learning behavior.

Claire discussed how bats respond to calling frogs when foraging. She also spoke about choice overload in that they make detrimental decisions when loaded with too many options.

Direct download: optimal-foraging.mp3
Category:general -- posted at: 3:02pm PST

On today’s show, we are joined by our co-host, Becky Hansis-O’Neil. Becky is a Ph.D. student at the University of Missouri, St Louis, where she studies bumblebees and tarantulas to understand their learning and cognitive work.

 

She joins us to discuss the paper: Perception in Chess. The paper aimed to understand how chess players perceive the positions of chess pieces on a chess board. She discussed the findings paper. She spoke about situations where grandmasters had better recall of chess positions than beginners and situations where they did not.

 

Becky and Kyle discussed the use of chess engines for cheating. They also discussed how chess players use chunking. Becky discussed some approaches to studying chess cognition, including eye tracking, EEG, and MRI. 

## Paper in Focus

Perception in chess

## Resources

Detecting Cheating in Chess with Ken Regan

Direct download: memory-and-chess.mp3
Category:general -- posted at: 9:33am PST

On this episode, we are joined by Stephen Larson, the CEO of MetaCell and an affiliate of the OpenWorm foundation. Stephen discussed what the Openworm project is about. They hope to use a digital C. elegans nematode (C. elegans for short) to study the basics of life.

Stephen discussed why C. elegans is an ideal organism for studying life in the lab. He also discussed the steps involved in simulating a digital organism. He mentioned the constraints on the cellular scale that informed their development of a digital C. elegans.

Stephen discussed the validation process of the simulation. He discussed how they discovered the best parameters to capture the behavior of natural C. elegans. He also discussed how biologists embraced the project.

Stephen discussed the computational requirements for improving the simulation parameters of the model and the kind of data they require to scale up. Stephen discussed some findings that the machine-learning communities can take away from the project. He also mentioned how students can get involved in the Openworm project. Rounding up, he shared future plans for the project.

Direct download: openworm.mp3
Category:general -- posted at: 3:38pm PST

Our guest is Becky Hansis-O’Neil, a Ph.D. student at the University of Missouri, St Louis, and our co-host for the new "Animal Intelligence" season. Becky shares her background on how she got into the field of behavioral intelligence and biology.

Direct download: what-the-antlion-knows.mp3
Category:general -- posted at: 4:13pm PST

Kyle is joined by friends and former guests Pramit Choudhary and Frank Bell to have an open discussion of the impacts LLMs and machine learning have had in the past year on industry, and where things may go in the current year.

Direct download: ai-roundtable.mp3
Category:general -- posted at: 1:52pm PST

We are joined by Darren McKee, a Policy Advisor and the host of Reality Check — a critical thinking podcast. Darren gave a background about himself and how he got into the AI space.

Darren shared his thoughts on AGI's achievements in the coming years. He defined AGI and discussed how to differentiate an AGI system. He also shared whether AI needs consciousness to be AGI.

Darren discussed his worry about AI surpassing human understanding of the universe and potentially causing harm to humanity. He also shared examples of how AI is already used for nefarious purposes. He explored whether AI possesses inherently evil intentions and gave his thoughts on regulating AI.

Direct download: uncontrollable-ai-risks.mp3
Category:general -- posted at: 11:05am PST

It took a massive financial investment for the first large language models (LLMs) to be created.  Did their corporate backers lock these tools away for all but the richest?  No.  They provided comodity priced API options for using them.  Anyone can talk to Chat GPT or Bing.  What if you want to go a step beyond that and do something programatic?  Kyle explores your options in this episode.

Direct download: i-llm-and-you-can-too.mp3
Category:general -- posted at: 11:51am PST

We celebrate episode 1000000000 with some Q&A from host Kyle Polich.  We boil this episode down to four key questions:

1) How do you find guests

2) What is Data Skeptic all about?

3) What is Kyle all about?

4) What are Kyle's thoughts on AGI?

 

Thanks to our sponsors
dataannotation.tech/programmers
https://www.webai.com/dataskeptic

 

Direct download: q-and-a-with-kyle.mp3
Category:general -- posted at: 8:55pm PST

In this episode, we are joined by Amir Netz, a Technical Fellow at Microsoft and the CTO of Microsoft Fabric. He discusses how companies can use Microsoft's latest tools for business intelligence.

Amir started by discussing how business intelligence has progressed in relevance over the years. Amir gave a brief introduction into what Power BI and Fabric are. He also discussed how Fabric distinguishes from other BI tools by building an end-to-end tool for the data journey.

Amir spoke about the process of building and deploying machine learning models with Microsoft Fabric. He shared the difference between Software as a Service (SaaS) and Platform as a Service (PaaS).

Amir discussed the benefits of Fabric's auto-integration and auto-optimization abilities. He also discussed the capabilities of Copilot in Fabric. He also discussed exciting future developments planned for Fabric. Amir shared techniques for limiting Copilot hallucination.

Direct download: llms-for-data-analysis.mp3
Category:general -- posted at: 10:03pm PST

Our guest today is Eric Boyd, the Corporate Vice President of AI at Microsoft. Eric joins us to share how organizations can leverage AI for faster development.

Eric shared the benefits of using natural language to build products. He discussed the future of version control and the level of AI background required to get started with Azure AI. He mentioned some foundational models in Azure AI and their capabilities. Follow Eric on LinkedIn to learn more about his work.

Visit today's sponsor at https://webai.com/dataskeptic

Direct download: ai-platforms.mp3
Category:general -- posted at: 6:33am PST

We are excited to be joined by Aaron Reich and Priyanka Shah. Aaron is the CTO at Avanade, while Priyanka leads their AI/IoT offering for the SEA Region. Priyanka is also the MVP for Microsoft AI. They join us to discuss how LLMs are deployed in organizations.

Direct download: deploying-llms.mp3
Category:general -- posted at: 10:28am PST

In this episode, we are joined by Jenny Liang, a PhD student at Carnegie Mellon University, where she studies the usability of code generation tools. She discusses her recent survey on the usability of AI programming assistants.

Jenny discussed the method she used to gather people to complete her survey. She also shared some questions in her survey alongside vital takeaways. She shared the major reasons for developers not wanting to us code-generation tools. She stressed that the code-generation tools might access the software developers' in-house code, which is intellectual property.

Learn more about Jenny Liang via https://jennyliang.me/

 

Direct download: a-survey-assessing-github-copilot.mp3
Category:general -- posted at: 11:11am PST

We are joined by Aman Madaan and Shuyan Zhou. They are both PhD students at the Language Technology Institute at Carnegie Mellon University. They join us to discuss their latest published paper, PAL: Program-aided Language Models.

Aman and Shuyan started by sharing how the application of LLMs has evolved. They talked about the performance of LLMs on arithmetic tasks in contrast to coding tasks. Aman introduced their PAL model and how it helps LLMs improve at arithmetic tasks. He shared examples of the tasks PAL was tested on. Shuyan discussed how PAL’s performance was evaluated using Big Bench hard tasks.

They discussed the kind of mistakes LLMs tend to make and how the PAL’s model circumvents these limitations. They also discussed how these developments in LLMS can improve kids learning.

Rounding up, Aman discussed the CoCoGen project, a project that enables NLP tasks to be converted to graphs. Shuyan and Aman shared their next research steps.

Follow Shuyan on Twitter @shuyanzhxyc. Follow Aman on @aman_madaan.

Direct download: program-aided-language-models.mp3
Category:general -- posted at: 7:00am PST

In this episode, we have Alessio Buscemi, a software engineer at Lifeware SA. Alessio was a post-doctoral researcher at the University of Luxembourg. He joins us to discuss his paper, A Comparative Study of Code Generation using ChatGPT 3.5 across 10 Programming Languages.  Alessio shared his thoughts on whether ChatGPT is a threat to software engineers. He discussed how LLMs can help software engineers become more efficient.

Direct download: which-programming-language-is-chatgpt-best-at.mp3
Category:general -- posted at: 6:00am PST

On the show today, we are joined by Jianan Zhao, a Computer Science student at Mila and the University of Montreal. His research focus is on graph databases and natural language processing. He joins us to discuss how to use graphs with LLMs efficiently.

 

Direct download: graph-text.mp3
Category:general -- posted at: 11:33am PST

Today, we are joined by Rajiv Movva, a PhD student in Computer Science at Cornell Tech University. His research interest lies in the intersection of responsible AI and computational social science. He joins to discuss the findings of this work that analyzed LLM publication patterns.

He shared the dataset he used for the survey. He also discussed the conditions for determining the papers to analyze. Rajiv shared some of the trends he observed from his analysis. For one, he observed there has been an increase in LLMs research. He also shared the proportions of papers published by universities, organizations, and industry leaders in LLMs such as OpenAI and Google. He mentioned the majority of the papers are centered on the social impact of LLMs. He also discussed other exciting application of LLMs such as in education.

Direct download: arxiv-publication-patterns.mp3
Category:general -- posted at: 10:20am PST

We are excited to be joined by Josh Albrecht, the CTO of Imbue. Imbue is a research company whose mission is to create AI agents that are more robust, safer, and easier to use. He joins us to share findings of his work; Despite "super-human" performance, current LLMs are unsuited for decisions about ethics and safety.

 

Direct download: do-llms-make-ethical-choices.mp3
Category:general -- posted at: 6:00am PST

On today’s show, we are joined by Thilo Hagendorff, a Research Group Leader of Ethics of Generative AI at the University of Stuttgart. He joins us to discuss his research, Deception Abilities Emerged in Large Language Models.

Thilo discussed how machine psychology is useful in machine learning tasks. He shared examples of cognitive tasks that LLMs have improved at solving. He shared his thoughts on whether there’s a ceiling to the tasks ML can solve.

Direct download: emergent-deception-in-llms.mp3
Category:general -- posted at: 8:26am PST

Nieves Montes, a Ph.D. student at the Artificial Intelligence Research Institute in Barcelona, Spain, joins us. Her PhD research revolves around value-based reasoning in relation to norms. She shares her latest study, Combining theory of mind and abductive reasoning in agent‑oriented programming.

Direct download: agents-with-theory-of-mind-play-hanabi.mp3
Category:general -- posted at: 6:58am PST

We are joined by Maximilian Mozes, a PhD student at the University College, London. His PhD research focuses on Natural Language Processing (NLP), particularly the intersection of adversarial machine learning and NLP. He joins us to discuss his latest research, Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities.

Direct download: llms-for-evil.mp3
Category:general -- posted at: 7:00am PST

Our guest today is Vid Kocijan, a Machine Learning Engineer at Kumo AI. Vid has a Ph.D. in Computer Science at the University of Oxford. His research focused on common sense reasoning, pre-training in LLMs, pretraining in knowledge-based completion, and how these pre-trainings impact societal bias. He joins us to discuss how he built a BERT model that solved the Winograd Schema Challenge.

Direct download: the-defeat-of-the-winograd-schema-challenge.mp3
Category:general -- posted at: 6:33am PST

Today, We are joined by Petter Törnberg, an Assistant Professor in Computational Social Science at the University of Amsterdam and a Senior Researcher at the University of Neuchatel. His research is centered on the intersection of computational methods and their applications in social sciences. He joins us to discuss findings from his research papers, ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning, and How to use LLMs for Text Analysis.

Direct download: llms-in-social-science.mp3
Category:general -- posted at: 9:25am PST

In this episode, we are joined by Carlos Hernández Oliván, a Ph.D. student at the University of Zaragoza. Carlos’s interest focuses on building new models for symbolic music generation.

Carlos shared his thoughts on whether these models are genuinely creative. He revealed situations where AI-generated music can pass the Turing test. He also shared some essential considerations when constructing models for music composition.

Direct download: llms-in-music-composition.mp3
Category:general -- posted at: 6:45am PST

Hongyi Wang, a Senior Researcher at the Machine Learning Department at Carnegie Mellon University, joins us. His research is in the intersection of systems and machine learning. He discussed his research paper, Cuttlefish: Low-Rank Model Training without All the Tuning, on today’s show.

Hogyi started by sharing his thoughts on whether developers need to learn how to fine-tune models. He then spoke about the need to optimize the training of ML models, especially as these models grow bigger. He discussed how data centers have the hardware to train these large models but not the community. He then spoke about the Low-Rank Adaptation (LoRa) technique and where it is used.

Hongyi discussed the Cuttlefish model and how it edges LoRa. He shared the use cases of Cattlefish and who should use it. Rounding up, he gave his advice on how people can get into the machine learning field. He also shared his future research ideas.

Direct download: cuddlefish-model-tuning.mp3
Category:general -- posted at: 7:38am PST

On today’s episode, we have Daniel Rock, an Assistant Professor of Operations Information and Decisions at the Wharton School of the University of Pennsylvania. Daniel’s research focuses on the economics of AI and ML, specifically how digital technologies are changing the economy.

Daniel discussed how AI has disrupted the job market in the past years. He also explained that it had created more winners than losers.

Daniel spoke about the empirical study he and his coauthors did to quantify the threat LLMs pose to professionals. He shared how they used the O-NET dataset and the BLS occupational employment survey to measure the impact of LLMs on different professions. Using the radiology profession as an example, he listed tasks that LLMs could assume.

Daniel broadly highlighted professions that are most and least exposed to LLMs proliferation. He also spoke about the risks of LLMs and his thoughts on implementing policies for regulating LLMs.

Direct download: which-professions-are-threatened-by-llms.mp3
Category:general -- posted at: 5:00am PST

We are excited to be joined by J.D. Zamfirescu-Pereira, a Ph.D. student at UC Berkeley. He focuses on the intersection of human-computer interaction (HCI) and artificial intelligence (AI). He joins us to share his work in his paper, Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts.  The discussion also explores lessons learned and achievements related to BotDesigner, a tool for creating chat bots.

Direct download: why-prompting-is-hard.mp3
Category:general -- posted at: 10:13am PST

In this episode, we are joined by Ryan Liu, a Computer Science graduate of Carnegie Mellon University. Ryan will begin his Ph.D. program at Princeton University this fall. His Ph.D. will focus on the intersection of large language models and how humans think. Ryan joins us to discuss his research titled "ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing"

Direct download: automated-peer-review.mp3
Category:general -- posted at: 7:00am PST

The creators of large language models impose restrictions on some of the types of requests one might make of them.  LLMs commonly refuse to give advice on committing crimes, producting adult content, or respond with any details about a variety of sensitive subjects.  As with any content filtering system, you have false positives and false negatives.

Today's interview with Max Reuter and William Schulze discusses their paper "I'm Afraid I Can't Do That: Predicting Prompt Refusal in Black-Box Generative Language Models".  In this work, they explore what types of prompts get refused and build a machine learning classifier adept at predicting if a particular prompt will be refused or not.

Direct download: prompt-refusal.mp3
Category:general -- posted at: 6:00am PST

Our guest today is Maciej Świechowski. Maciej is affiliated with QED Software and QED Games. He has a Ph.D. in Systems Research from the Polish Academy of Sciences. Maciej joins us to discuss findings from his study, Deep Learning and Artificial General Intelligence: Still a Long Way to Go.

Direct download: a-long-way-till-agi.mp3
Category:general -- posted at: 4:00am PST

Today on the show, we are joined by Lin Zhao and Lu Zhang. Lin is a Senior Research Scientist at United Imaging Intelligence, while Lu is a Ph.D. candidate at the Department of Computer Science and Engineering at the University of Texas. They both shared findings from their work When Brain-inspired AI Meets AGI.

Lin and Lu began by discussing the connections between the brain and neural networks. They mentioned the similarities as well as the differences. They also shared whether there is a possibility for solid advancements in neural networks to the point of AGI. They shared how understanding the brain more can help drive robust artificial intelligence systems.

Lin and Lu shared how the brain inspired popular machine learning algorithms like transformers. They also shared how AI models can learn alignment from the human brain. They juxtaposed the low energy usage of the brain compared to high-end computers and whether computers can become more energy efficient.

Direct download: brain-inspired-ai.mp3
Category:general -- posted at: 5:45pm PST

On today’s show, we are joined by Michael Timothy Bennett, a Ph.D. student at the Australian National University. Michael’s research is centered around Artificial General Intelligence (AGI), specifically the mathematical formalism of AGIs. He joins us to discuss findings from his study, Computable Artificial General Intelligence.

Direct download: computable-agi.mp3
Category:general -- posted at: 6:00am PST

We are joined by Koen Holtman, an independent AI researcher focusing on AI safety. Koen is the Founder of Holtman Systems Research, a research company based in the Netherlands.

Koen started the conversation with his take on an AI apocalypse in the coming years. He discussed the obedience problem with AI models and the safe form of obedience.

Koen explained the concept of Markov Decision Process (MDP) and how it is used to build machine learning models.

Koen spoke about the problem of AGIs not being able to allow changing their utility function after the model is deployed. He shared another alternative approach to solving the problem. He shared how to engineer AGI systems now and in the future safely. He also spoke about how to implement safety layers on AI models.

Koen discussed the ultimate goal of a safe AI system and how to check that an AI system is indeed safe. He discussed the intersection between large language Models (LLMs) and MDPs. He shared the key ingredients to scale the current AI implementations.

Direct download: agi-can-be-safe.mp3
Category:general -- posted at: 1:09pm PST

An assistant professor of Psychology at Harvard University, Tomer Ullman, joins us. Tomer discussed the theory of mind and whether machines can indeed pass it. Using variations of the Sally-Anne test and the Smarties tube test, he explained how LLMs could fail the theory of mind test.

Direct download: ai-fails-on-theory-of-mind-tasks.mp3
Category:general -- posted at: 9:35am PST

The application of LLMs cuts across various industries. Today, we are joined by Steven Van Vaerenbergh, who discussed the application of AI in mathematics education. He discussed how AI tools have changed the landscape of solving mathematical problems. He also shared LLMs' current strengths and weaknesses in solving math problems.

Direct download: ai-for-mathematics-education.mp3
Category:general -- posted at: 6:00am PST

Fabricio Goes, a Lecturer in Creative Computing at the University of Leicester, joins us today. Fabricio discussed what creativity entails and how to evaluate jokes with LLMs. He specifically shared the process of evaluating jokes with GPT-3 and GPT-4. He concluded with his thoughts on the future of LLMs for creative tasks.

Direct download: evaluating-jokes-with-llms.mp3
Category:general -- posted at: 9:06am PST

Barry Smith and Jobst Landgrebe, authors of the book “Why Machines will never Rule the World,” join us today. They discussed the limitations of AI systems in today’s world. They also shared elaborate reasons AI will struggle to attain the level of human intelligence.

Direct download: why-machines-will-never-rule-the-world.mp3
Category:general -- posted at: 3:17pm PST

While the possibilities with AGI emergence seem great, it also calls for safety concerns. On the show, Vahid Behzadan, an Assistant Professor of Computer Science and Data Science, joins us to discuss the complexities of modeling AGIs to accurately achieve objective functions. He touched on tangent issues such as abstractions during training, the problem of unpredictability, communications among agents, and so on.

Direct download: a-psychopathological-approach-to-safety-in-agi.mp3
Category:general -- posted at: 11:59pm PST

Julian Michael, a postdoc at the Center for Data Science, New York University, joins us today. Julian’s conversation with Kyle was centered on the NLP community metasurvey: a survey aimed at understanding expert opinions on controversial NLP issues. He shared the process of preparing the survey as well as some shocking results.

Direct download: the-nlp-community-metasurvey.mp3
Category:general -- posted at: 6:00am PST

Kyle shares his own perspectives on challenges getting insight from surveys. The discussion ranges from commentary on the market research industry to specific advice for detecting disingenuous or fraudulent responses and filtering them from your analysis. Finally, he shares some quick thoughts on the usage of the Chi-Square test for interpreting cross tab results in survey analysis.

 
Direct download: skeptical-survey-interpretation.mp3
Category:general -- posted at: 5:10pm PST

Jeff Jones, a Senior Editor at Gallup, joins us today. His conversation with Kyle spanned a range of topics on Gallup’s poll creation process. He discussed how Gallup generates unbiased questionnaires, gets respondents, analyzes results, and everything in between.

Direct download: the-gallup-poll.mp3
Category:general -- posted at: 9:49am PST

Gireeja Ranade, a University of California at Berkeley professor, speaks with us today. She presented her study on implementing inclusive study groups at scale and shared the observed student performance improvements after the intervention.

Direct download: inclusive-study-group-formation-at-scale.mp3
Category:general -- posted at: 9:34pm PST

Today, we are joined by David Bourget. David is an Associate Professor in Philosophy at Western University in London, Ontario. David is also the co-director of the PhilPapers Foundation and Director of the Center for Digital Philosophy. He joins us to discuss the PhilPapers Survey project.

The PhilPapers survey was initially taken in 2009, but there was a follow-up survey in 2020. David discussed the need for the subsequent survey and what changed. He mentioned the metric for measuring the opinion changes between the 2009 and 2020 surveys. He also shared future plans for the PhilPapers surveys.

Direct download: the-phil-papers-survey.mp3
Category:general -- posted at: 11:51am PST

Today’s show focused on an essential part of surveys — missing values. This is typically caused by a low response rate or non-response from respondents. Yajuan Si is a Research Associate Professor at the Survey Research Center at the University of Michigan. She joins us to discuss dealing with bias from low survey response rates.

Direct download: non-response-bias.mp3
Category:general -- posted at: 10:00am PST

We are joined by two guests today, Mariah, a Ph.D. student in the CORE Robotics Lab at Georgia Tech, and Matthew Gombolay, the Director of the CORE Robotics Lab. They both discuss practices for measuring a respondent’s perception in a survey.

Direct download: measuring-trust-in-robots-with-likert-scales.mp3
Category:general -- posted at: 11:05am PST

Ever wondered what your next career would be? Today, Keyon Vafa, a computer science Ph.D. student at Columbia University, joins us to discuss his latest research on developing a machine-learning model for career prediction. Keyon extensively spoke about how the model was developed and the possibilities it brings.

Direct download: career-prediction.mp3
Category:general -- posted at: 8:36am PST

Noura Insolera, a Research Investigator with the Panel Study of Income Dynamics (PSID), joins us to share how PSID conducts longitudinal household surveys. She also shared some interesting findings from their data exploration, particularly on the observation and trends in food insecurity.

Direct download: the-panel-study-of-income-dynamics.mp3
Category:general -- posted at: 11:13pm PST

Susan Gerbic joins Kyle to review some of the surveys Data Skeptic has launch, draft a new survey about podcast listening habits, and then review the results of that survey. You can see those results at the link below.

https://survey.dataskeptic.com/survey/result/1675102237053

Watch the videos Susan mentioned on her Youtube page at the link below.

https://www.youtube.com/playlist?list=PL7VAuaQDhPTVaLeI1IcpYph5lH19xA1u4

Direct download: survey-design-working-session.mp3
Category:general -- posted at: 11:10am PST

The use of social bots to fill out online surveys is becoming prevalent. Today, we speak with Sara Bybee, a postdoctoral research scholar at the University of Utah. Sara shares from her research, how she detected social bots, the strategies to curb them, and how underrepresented groups can be more represented in surveys.

Direct download: bot-detection-and-dyadic-surveys.mp3
Category:general -- posted at: 10:08am PST

Our guest today is Zoltán Kekecs, a Ph.D. holder in Behavioural Science. Zoltán highlights the problem of low replicability in journal papers and illustrates how researchers can better ensure complete replication of their research and findings. He used Bem’s experiment as an example, extensively talking about his methodology and results.

Direct download: reproducible-esp-testing.mp3
Category:general -- posted at: 6:00am PST

On the show, Iñigo Martinez, a Ph.D. student at the University of Navarra shares his survey results which investigated how data practitioners perform data science projects. He revealed the methodologies typically used by data practitioners and the success factors in data science projects.

Direct download: a-survey-of-data-science-methodologies.mp3
Category:general -- posted at: 9:43am PST

On the show today, Dino Carpentras, a post-doctoral researcher at the Computational Social Science group at ETH Zürich joins us to discuss how opinion dynamics models are built and validated. He explained how quantifying opinions is complex, and strategies to develop robust models for measuring and predicting public opinions.

Direct download: opinion-dynamics-models.mp3
Category:general -- posted at: 12:17pm PST

Crafting survey questions is one thing but getting your audience to fill it is yet another. On the show today, we speak with Alexander Nolte, an Associate Professor at the University of Tartu. Alexander discussed the use of Casual Affective Triggers (CAT) to incentivize people to accept survey invitations and improve the completion rate. He revealed the impact of CATs on survey response rates from a study he conducted.

Direct download: causal-affective-triggers.mp3
Category:general -- posted at: 11:58am PST

Traditional surveys have straight-jacket questions to be answered, thus restricting the information that can be gotten. Today, Ziang Xiao, a Postdoc Researcher in the FATE group at Microsoft Research Montréal, talks about conversational surveys, a type of survey that asks questions based on preceding answers. He discussed the benefits of conversational surveys and some of the challenges it poses.

Direct download: conversational-surveys.mp3
Category:general -- posted at: 6:00am PST

Today, Jenny Tang, a Ph.D. student of societal computing at Carnegie Mellon University discusses her work on the generalization of privacy and security surveys on platforms such as Amazon MTurk and Prolific. Jenny shared the drawbacks of using such online platforms, the discrepancies observed about the samples drawn, and key insights from her results.

Direct download: do-results-generalize-for-privacy-and-security-surveys.mp3
Category:general -- posted at: 12:39pm PST

This episode kicks off the new season of the show, Data Skeptic: Surveys.  Linhda rejoins the show for a conversation with Kyle about her experience taking surveys and what questions she has for the season.  Lastly, Kyle announces the launch of survey.dataskeptic.com, a new site we're launching to gather your opinions.  Please take a moment and share your thoughts!

Direct download: 4-out-of-5-data-scientists-agree.mp3
Category:general -- posted at: 8:46am PST

It may be intuitive to think crowdfunding a project drives its innovation and novelty, but there are no empirical studies that prove this. On the show, Johannes Wachs shares his research that sought to determine whether crowdfunding truly drives innovation. He used board games as a case study and shared the results he found.

Direct download: crowdfunded-board-games.mp3
Category:general -- posted at: 6:00am PST

There were reports of Russia’s interference in the 2016 US elections. In today’s episode, Koustuv Saha, a researcher at Microsoft Research walks us through the effect of targeted ads for political campaigns. Using practical examples, he discusses how targeted ads can propagate fake news, its ripple effects on electioneering, and how to find a sweet spot with targeted ads.

Direct download: russian-election-interference-effectiveness.mp3
Category:general -- posted at: 6:05am PST

There is an unsung kind of ad fraud brewing in the ad tech space — placement laundering fraud. On the show, Jeff Kline discusses what placement laundering fraud is, how it can be identified, and possible solutions to it. Listen to learn more.

Direct download: placement-laundering-fraud.mp3
Category:general -- posted at: 9:38am PST

Bosko Milekic, the Co-founder of Optable, a data collaboration platform for the media and advertising industry, joins us today. Bosko talked about the clean rooms, the technology driving data privacy during collaboration. He discussed why clean rooms are gaining widespread adoption, and how users can exploit Optable’s clean room platform for a secured data-sharing experience.

Direct download: data-clean-rooms.mp3
Category:general -- posted at: 9:54am PST

Kerstin Bongard-Blanchy is a Research Associate at the University of Luxembourg. She joins us to discuss her study that investigated dark patterns in web designs. She discussed the results, the effect of dark patterns effect on users, whether an average user can detect them, and the way forward to a more ethical web space.

Direct download: dark-patterns-in-site-design.mp3
Category:general -- posted at: 9:04am PST

We are joined by Anthony Katsur, the CEO of IAB Tech Lab. Anthony discusses standards within the ad tech industry. He explained how IAB Tech Lab set and propagates global standards, actions to ensure compliance from advertisers, and industry trends for a more privacy-centric ad tech space.

Direct download: internet-advertising-bureau-media-lab.mp3
Category:general -- posted at: 9:19am PST

When we navigate a webpage, it is fairly easy for our mouse movement to be tracked and collected. Today, Luis Leiva, a Professor of Computer Science discusses how these mouse tracking data can be used to predict age, gender and user attention. He also discusses the privacy concerns with mouse tracking data and possible ways it can be curtailed.

Direct download: your-mouse-reveals-your-gender-and-age.mp3
Category:general -- posted at: 6:00am PST

On the show, Aleksandra Urman and Mykola Makhortykh join us to discuss their work on the comparative analysis of web search behavior using web tracking data. They shared interesting results from their analysis, bordering around the user preferences for search engines, demographic patterns, and differences between how men and women surf the net.

Direct download: measuring-web-search-behavior.mp3
Category:general -- posted at: 7:56am PST

Did Aristotle Use a Laptop?  That's a question from the StrategyQA benchmark which highlights the stretch goals for current artificial intelligence systems.  Answering a question like that requires several cognitive steps and reasoning.  Constructing a dataset of similarly challenging questions is a major undertaking.  On today's episode, Mor Geva returns to share details about the creation of StrategyQA and the larger Big Bench dataset it has been included in.

Direct download: big-bench.mp3
Category:general -- posted at: 7:39pm PST