1/18/2018 Lecture Notes
January 18, 2018
Welcome back! Professor Pacheco is out of town today. You should begin the class working on Assignment 2, which is to find some data from the New York State data site and tell a story with it using Instagram.
At 1 p.m., Professor Jodi Upton, our Knight Chair in Data and Explanatory Journalism, will arrive to give a guest lecture about data journalism.
January 16, 2018
1/16/2018 Lecture Notes: Welcome to Dataviz!
January 11, 2018
Tues 1/16/2018 Class: Welcome!
12:30-12:50: Welcome! An Introduction to using data visualization to tell stories.
- Quick roll call.
- Before next class, please register for the class blog (follow instructions from an email the system sent you) and fill out this survey.
- Review how the class site is organized.
- Walkthrough of the syllabus and schedule.
12:50-1: What is Dataviz?
- Some things can be better understood by seeing or exploring. Here are some good examples:
- News you can Use: Is it better to buy or rent?
- Understanding complexity. Which Supreme Court justices agree or disagree?
- Understanding behaviors. Where do voters in NYC live?
- Understanding sentiment. Emotional arcs during presidential addresses.
- Understanding geographic distribution. Perentage of people without health insurance by U.S. county.
- Understanding our world. http://Hint.fm/wind
- Having fun! Here are some visualizations about where in the world people have the best and worst sex and recurring themes in Arrested Development.
- More examples: Here are some more I have collected, and on the #Dataviz Twitter hashtag and Data is Beautiful Reddit.
- And how about some really bad examples!
1-1:15: Exercise: And now for a little magic. Introduce yourself through data!
- Open this URL. How did that data get in there? Let’s find out together.
- Google yourself and find an image that is publicly available on the web. Right-click the image and get the URL.
- Go to this Google spreadsheet and fill out your info. Put your image URL into the correct column, and put this code around it:
<img src=”YOURURLHERE” width=”200″ />
(Note: type this in, don’t copy and paste from this page).
- Take a look at this URL or the home page to see class data populating in real time.
Congrats! You Participated in a Data Visualization
We will go through some basic features of Excel, and formulas.
- Adding information as data
- Add a formula
- Columns and rows.
- Formula: using the equal sign for functions. Basic math.
- Sum columns or rows.
- Select an area.
- Format cells to change cell type (text, number).
- Making charts in Excel.
- Common formulas: adding, subtracting, dividing, multiplying, summing.
- CSV format versus native Excel format.
1:25-1:40: Putting it into practice: Sorting and filtering NYS bridge data
- Bridges across the country are badly in need of repair, and it can literally be a life or death issue. Here’s more about that.
- Download and unzip this data set of 51,000 bridges in New York State. bridges_blanksremoved.csv
- Open it in Excel. Scroll right until you find the column “critfrac” (column DL) which stands for critical fracture. A y12 or y24 means outdated design, so a single solid hit can bring the entire bridge down.
- Next, find the column “suffrtno” (column FC), which stands for Sufficiency Rating. Anything under 50 is considered dangerous.
- Also note the “totlcost” (total cost to fix in thousands of dollars) in column DV, and “avdayno” (average daily traffic) in column AK.
- How many bridges are in danger of collapsing due to critical fracture?
- How many bridges have an inadequate sufficiency rating?
- How many have both bad critical fracture and sufficiency rating numbers?
- How much traffic goes over the bridges with both bad critfrac and suffrtno ratings? (Use the data from “avdayno,” column AK).
- How much will it cost to fix the bridges with both bad critfrac and suffrtno ratings? (Use “totlcost”, column DV).Go through these yourself, then let’s review the answers and how to get them.
1:40-1:50: Looking ahead:
5/2 Lecture Notes
May 2, 2017
12:30-12:40: Field test and Vision Paper
- Both the field test and vision paper are due Thursday, May 11 midnight.
- The vision “paper” is not a paper, it’s a a short scifi story about what your media career looks like 5 to 10 years in the future as influenced by the technologies we have covered.
- Here are a couple examples:
12:40-1:10: Sensor Journalism Presentation
1:10-1:15: Arduino demo
1:15-1:50: Assignment 5
- Read through one of the Case Studies section of this report, beginning on page 45 (see link in #NTNM hashtag): http://towcenter.org/wp-content/uploads/2014/05/Tow-Center-Sensors-and-Journalism.pdf
- Go to http://sparkfun.com and click on Sensors. Try to find the sensors mentioned in these case studies. Then, come up with a concept for a theoretical sensor array that could be used in a journalistic story.
- Post about both in a blog entry.
Remember, as mentioned in the syllabus, we are required to meet one last time and that is scheduled for Thursday at the same class time as usual. If you can’t make it because of a final exam, consider that your “one class you can miss for any or no reason.” If you can make it, we will meet in the innovation lab and you can use the time to work on your field test.
4/20/2017 Lecture Notes
April 20, 2017
On Tuesday we started learning how to create chatbots. Here are links to the info and services we went over.
- Try the TechCrunch Facebook Messenger bot: http://m.me/TechCrunch
- Try Dan Pacheco’s autodanpacheco bot: m.me/autodanpacheco
- Create your own Chatbot: http://chatfuel.com
- Remember that you will need to create a Facebook page for the bot before you can publish it. You can create that here: facebook.com/pages/create
Your Third assignment is to create a Facebook Messenger chat bot using Chatfuel.com. The topic of the bot should be something informational. This could be a topic in the news, information about an event (future or historical), a science topic, information about Newhouse, etc.
12:40-1:10: Amazon Skills
Today we will look into voice-based assistants, specifically Amazon’s Alexa and Alexa “Skills,” which are like voice-activated apps for the Amazon Echo.
- Demo of how Alexa works.
- Installing Alexa Skills.
- Creating a conversation tree with PullString, and (theoretically) how to publish that as a voice-activated Alexa Skill. You can also use PullString to create a basic chat bot for other services.
- You can also try creating an Alexa skill using Amazon’s new Skills kit and its Builder Beta tool.
- Quick demo of http://IFTTT.com
4/11/2017 Lecture Notes
April 11, 2017
12:30: I’m back!
- International Journalism Festival report.
Watch the session here.
- See my 360 videos of Assisi and Perugia, two previously warring city-states in Italy.
- How 360 in YouTube works with Cardboard (send out link, everyone views in cardboard)
12:55: Walk to Innovation lab
1:05-1:50: 360 video
- Shooting on Samsung Gear360 and Nikon KeyMission
- Transferring videos
- Stitching videos (Samsung only, Nikon is automatic)
- Processing and exporting in Adobe Premiere Pro
- Adding 2D text effects in SkyBox VR Tools plugin
—> You can access this software in the lab, or in 206B or 205A.
- Only 206B has Skybox.
- You can shoot a 360 video with a team, or …
- Learn how to create computer-generated environments in Unity
4/4/2017 Lecture Notes – welcome!
April 4, 2017
Welcome! Here’s the presentation from the first class.
Assignment 5: Sortable, searchable list
March 23, 2017
Use Tabletop to create a sortable, searchable list for a story using data you find on your own. Upload it to the class FTP and embed the list in your story.
In addition, summarize a subset of the data in a chart using one of the other tools we have covered so far in class such as Infogr.am, Google Maps, Highcharts.
Due: Tuesday, March 28 before class.
Assignment 2: Sorting and Filtering in Excel
February 23, 2017
In this exercise we will sort and filter data to understand it, then interviewing the data to gain insights and, hopefully, tell simple stories.
Data Viz – Sorting and Filtering Exercise
Using New York State Cardiac Arrests
New York State has an open-data policy. You can get a lot of data from New York State-funded programs at https://data.ny.gov/.
In this exercise, we will be using data on salary information for New York State authorities: https://data.ny.gov/Transparency/Salary-Information-for-State-Authorities/unag-2p27
Click the Export button and choose CSV for Excel, then save to your desktop.
Let’s make a list of the biggest fat cats in state government. Go to the Data menu and choose Sort.
- Make sure the box labeled “My list has headers” is checked!
- Click in “Sort By” and choose “Base Annual Salary” (column L). Under “Order” choose “Largest to Smallest,” then click OK.
- Who’s making the big money?
- Change the sorting to smallest to largest. Who’s making the least?
- Do you see any trends in the types of people who make the most and least amount of money?
- Do you see any problems with missing data?
As you may notice, there are some records that have $0 for salaries, so we need to filter those out.
- Go to the Data menu and choose Filter. You should see downward-facing arrows above each column header.
- Click the arrow in the Base Annual Salary column.
- Uncheck the box next to $0.
Now, let’s remove all salaries that are below the Federal poverty line of $20,000. Create a Sort, and sort on Base Annual Salary from smallest to largest. Select all of the rows under $20,000 and choose Edit > Delete Row.
Finally, let’s use a combination of filtering, sorting and formulas to find the average salary of people in the “Managerial” group (column H).
Now you should be familiar with sorting and filtering. What trends do you notice in the data? Do you see anything that may suggest a potential story? Or did you find a good story?
Now let’s shift to a topic that’s in the news a lot lately: how much money U.S. presidential candidates are getting. The Federal Election Commission makes this available in spreadsheet form here: http://www.fec.gov/disclosurep/PDownload.do
Download the data from the state where you’re from (or if not from the U.S. you can just choose New York). Use Sorting and Filtering to create a list of donations to two candidates you follow. When you create each list, use the Excel SUM feature to tally up how much money the candidate has received.
As a final step, create a simple bar graph in Infogr.am showing how much they’ve received. In class I will show you how to embed this in a blog post.
The assignment: Find some interesting data from the New York State open data site. Use sorting and filtering to hone in on some interesting and easily digestible data points that could be used in a story. Create a graph of the data you find in Infogr.am. Create a blog post that includes a link to the raw data from the NYS site, explain how you filtered it, and embed the Infogr.am chart into the post. Due Monday morning.
Assignment 1: Class survey and blog registration
February 22, 2017
Assignment 1 is to complete the registration for the class blog and fill out this survey. Both are due by Monday morning.