HomeОбразованиеRelated VideosMore From: Corey Schafer

Python Tutorial: CSV Module - How to Read, Parse, and Write CSV Files

3885 ratings | 258440 views
In this Python Programming Tutorial, we will be learning how to work with csv files using the csv module. We will learn how to read, parse, and write to csv files. CSV stands for "Comma-Separated Values". It is a common format for storing information. Knowing how to read, parse, and write this information to files will open the door to working with a lot of data throughout the world. Let's get started. The code from this video can be found at: https://github.com/CoreyMSchafer/code_snippets/tree/master/Python-CSV ✅ Support My Channel Through Patreon: https://www.patreon.com/coreyms ✅ Become a Channel Member: https://www.youtube.com/channel/UCCezIgC97PvUuR4_gbFUs5g/join ✅ One-Time Contribution Through PayPal: https://goo.gl/649HFY ✅ Cryptocurrency Donations: Bitcoin Wallet - 3MPH8oY2EAgbLVy7RBMinwcBntggi7qeG3 Ethereum Wallet - 0x151649418616068fB46C3598083817101d3bCD33 Litecoin Wallet - MPvEBY5fxGkmPQgocfJbxP6EmTo5UUXMot ✅ Corey's Public Amazon Wishlist http://a.co/inIyro1 ✅ Equipment I Use and Books I Recommend: https://www.amazon.com/shop/coreyschafer ▶️ You Can Find Me On: My Website - http://coreyms.com/ My Second Channel - https://www.youtube.com/c/coreymschafer Facebook - https://www.facebook.com/CoreyMSchafer Twitter - https://twitter.com/CoreyMSchafer Instagram - https://www.instagram.com/coreymschafer/ #Python
Html code for embedding videos on your blog
Text Comments (310)
David Lessnau (1 day ago)
Old video, but I don't see this in the comments. Shouldn't you be using something like random.seed(time) near the top? Otherwise, the sequence of random numbers you generate will be the same from run to run.
David Lessnau (1 day ago)
+Corey Schafer Yep. No idea how I managed that. But, I thought I was adding a comment in this video: https://youtu.be/KzqSDvzOFNA Sorry. Not sure if I should delete my OP and copy it there or just leave it here.
Corey Schafer (1 day ago)
Did you mean to comment this on another video? I'm not using the random module in this vid.
Baburam Chaudhary (2 days ago)
If we want to extract certain name or email defined by user. How to script it. Please help...
RAHUL KUMAR (4 days ago)
sir my question is that when we are entering some data in the registration form then that data. how to update in CSV file using python
Panduranga NM (8 days ago)
how to display python output in big display
S.M.Ehsanul Amin (13 days ago)
can you share this things for importing CSV files in mysql from a directory by using python while using load infile command ?
azri ridha (13 days ago)
Hello, very amzing vidéo 📼. I think its so easy to manage csv file now. Tank you alot. I have an value error : I/Ô opération on closed file when i run this code, can anyone help me plz
Okan Ozdemir (14 days ago)
Sir Archelot like Duke, please teach me pybibd 11 extend c++ and use python script with easy cmake. That will renew my birth
Nani nani (14 days ago)
hi .. i want python code for csv file to plots and histogram plots the file is (400001x16)
John Doe (15 days ago)
Excellent video as always. Question though? I placed each command in a function ex: def main(): def readCsv(): <Opens the csv and prints output to stdout as expected> def writeCsv(): <does not write to a file> main() in Laymans terms, could I not place both of these in separate functions? I want to leave them in functions in case I import them later. Thanks again for the great content!
umme farwa (20 days ago)
please make a video on converting specific rows from csv file to text
Albert Batfinder (22 days ago)
Corey, the only tricky thing about csv is when you have to deal with special characters within data fields. I don’t think you demonstrated how to handle quotes within strings, or other escape characters such as newlines. Also, it is really bad in an instructive piece of code to use variable names that too closely mimic language keywords. Fieldnames=fieldnames, for example. Or csv_reader = csv.reader(...).
rootGR (23 days ago)
I see you now and explain the python very well..
Literally Razor (25 days ago)
How do you even write one in the first place...
WhiteWolf (1 month ago)
Isn't it better to use SQL rather than CSV?
WhiteWolf (1 month ago)
Also can you do the same operations with .txt file, actually you can take it line by line and split it to get name,adress, second name etc. Why to use CSV?
Madhu Sibi (1 month ago)
Hello, I loved your video. I was wondering for instance if I have a to remove all the fields other than the first three in a file and store the first two as the key pair and the third as its value.. how do I do it?
Daniel Tkach (1 month ago)
The for loop is reading CHARACTERS instead of lines, so I see just one character when I print it. I guess this changed in 2019?
litty committee (1 month ago)
how do I get my sublime text to look like ur setup
Corey Schafer (1 month ago)
I made a video on how I set up this Python development environment. You can find that video here: https://www.youtube.com/watch?v=xFciV6Ew5r4 Thanks!
liwen huang (1 month ago)
Hello Corey, will the Dict method be faster or slower than the regular csv method?
Sandeep Chaurasiya (1 month ago)
how i can store the values entered in the html form into the current csv file using above mentioned method.????
Security Hub (1 month ago)
Your introduction voice is amazing!
Sean Smyth (2 months ago)
Hi Corey. Great video. In the video you mention a way to parse the csv columns by creating a new dictionary rather than the del method. I have a read file with about 100 columns but only need to write 10 of them could you show the alternate solution?
hank Cooper. (2 months ago)
does not work in PyCharm 4.05
Nikki Luzader (2 months ago)
if you don't use the newline parameter, writing the data to a new file will result in every other line being skipped. to alleviate this, use the newline parameter newline='' that is newline with two single quotes with open('new_names.csv','w', newline='') as new_file:
Peter Navarro III (2 months ago)
Nice, was wondering why it was adding an empty row on every other line.
Lightning in My Hands (2 months ago)
Just what I was looking for!
Nikki Luzader (2 months ago)
tinnick great question. I am actually not 100% sure to be perfectly honest as I am not an expert. The only logical explanation I can come up with is that the csv module has been updated since the video and the newline parameter was introduced in one of the updates. Modules are constantly being updated, as well as their default behavior. It’s just part of programming. Sometimes this can even break your already perfect application, leaving you to read through updated documentation to see what has changed. I hope this helps you.
tinnick (2 months ago)
Hi Nikki, I haven't really tried this yet so I don't clearly understand but, I wanted to know why this happens when writing the data. In the video the newline parameter was not passed but did not skip any lines. In what situation would it be skipped and what is the reasoning?
Nikki Luzader (2 months ago)
Charlie Skinner no problem!
Nico Scholz (2 months ago)
Hi i've exactly copied your code but when i let python read the "written new" csv file as you explain at around 10:00 into the video it prints "empty" lines in between the entrys. Has anybody got an idea whats wrong there? The file seems identical to my original file. Using python3, atom, win7 ['jane', ' doe', ' [email protected]'] [] ['john', ' mayer', '[email protected]'] []
Nikki Luzader (2 months ago)
the csv module has been updated since he created this video. you must use a new parameter when you call with open()... the additional parameter is called "newline" if you don't use the newline parameter, writing the data to a new file will result in every other line being skipped. to alleviate this, use the newline parameter newline='' that is newline with two single quotes example: with open('new_names.csv','w', newline='') as new_file:
Rihan Ansari (2 months ago)
❤️👏🏻
Andy Wong (2 months ago)
How do you do that multiple line comment at 9:36? It's a nice trick!
Jhon Jairo Díaz Garzón (2 months ago)
He used Sublime Text software to code... Jus't select the lines you want to comment and press #. Pardon for my english.
Mehak Verma (2 months ago)
I have got few issues and i am not getting any solution online, so would need your help. Anyways, first issue is i have 7 fields in csv file but output comes out just for 6 though header shows 7 names. If i don't use extrasaction = 'ignore', then i am getting below error, here is the snippet: import csv with open('test.csv','r+') as rf: csv_reader = csv.DictReader(rf) with open('new_test.csv','w+') as wf: field_names = ['Count', 'Time', 'Thread_ID', 'Process_ID', 'Session_ID', 'Module', 'Message'] csv_writer = csv.DictWriter(wf, fieldnames = field_names, extrasaction = 'ignore', delimiter = '\t') csv_writer.writeheader() for line in csv_reader: csv_writer.writerow(line) OUTPUT --> No value for Count key Count Time Thread_ID Process_ID Session_ID Module Message 11:06:02:444 3896 1456 230 CDF_ENTRY DllMain: enter 11:06:02:444 3896 1456 340 CDF_ENTRY DllMain: exit 11:06:02:444 3896 1456 750 CDF_ENTRY Exiting ERROR Traceback (most recent call last): File "D:\Projects\Python\FindIndex.py", line 11, in <module> csv_writer.writerow(line) File "C:\Users\mehakv\AppData\Local\Programs\Python\Python37-32\lib\csv.py", line 155, in writerow return self.writer.writerow(self._dict_to_list(rowdict)) File "C:\Users\mehakv\AppData\Local\Programs\Python\Python37-32\lib\csv.py", line 151, in _dict_to_list + ", ".join([repr(x) for x in wrong_fields])) ValueError: dict contains fields not in fieldnames: 'Count'
Ole Kristian Rannekleiv (2 months ago)
How do I read the CSV file line by line rather than row downwards? say I have a CSV file where the description of each row is on the first line, how do I see it ?
Nikki Luzader (2 months ago)
can you explain this a little better?
Rico MC (2 months ago)
Merci
Noga Kahana (2 months ago)
Is there is an advantage for using CSV format over pandas?
Nikki Luzader (2 months ago)
#1 csv functionality is built into python, pandas is not #2 it is more light weight and faster (but we are talking milliseconds) #3 pandas might be overkill depending on what you're doing. you might just want to gently manipulate csv files.
Corey Schafer (2 months ago)
None that I know of. If you're using Pandas for csv then that works too.
Rohit (3 months ago)
How to create a search engine after uploading in database online ,,,,,
Vegito_ Blue (3 months ago)
Hi Corey, thanks for this video. For my interview tomorrow this is what I was looking for.
Suraj Singh (3 months ago)
Thanks a Lottttttt!!!!!!!!!!!!!!!!!!!
Bruce Campbell (3 months ago)
I am using a tab-delimited text file, but for some reason, it is not changing when I specify other parameters. I want to modify only the 3rd column and there are no headers. Can you help? Your videos are wonderful!!! Thank you.
Farzan M. Noori (3 months ago)
You're a life saver :D <3
sheethal gowda (3 months ago)
can i change the the filednames to [first,last,email ]
Rajan Kumar (3 months ago)
where can i get CSV files problem to practice
Nikki Luzader (2 months ago)
https://www.mockaroo.com/
Hasif Khan (3 months ago)
I did it with downloading my google contacts export option.
Amanuel Siyum (3 months ago)
Very helpful videos, keep up the good work!!!!!
Wing Poon (3 months ago)
Seems redundant that DictWriter needs to be ~explicitly~ informed of the field names when it can discover it for itself, from the OrderedDict it is passed each time. Anyway, if you pass " extrasaction='ignore' " into csv.DictWrite(), you need not perform the del on the dict on every row ... seems like a cleaner way.
Chad Hanson (3 months ago)
when I try this by using your names file by copying the data into excel and saving as a CSV comma delimited file, my new names file adds a new line in between each row of my output?
Nikki Luzader (2 months ago)
there is a new parameter in the updated version of the csv module. just put newline=' ' (that is two single quotes, no space between) with open('new_names.csv','w', newline='') as new_file:
Ragib Shahariar (3 months ago)
one video one subscriber.
Ahmad Ali (3 months ago)
thanks
Neo Neos (3 months ago)
you are the best
roger vilardebo (4 months ago)
gracias parse! sin tu ayuda no hubieramos conseguido hacer el juego de la vida "parse.csv"
Andres Beltre (4 months ago)
Thank you so much for the sucint tutorial. This precisely what I needed!
Emmanuel Irog-irog (4 months ago)
I really need help on this I have two csv files dataA.csv and dataB.csv 1 2 2 3 3 5 4 2 how do I merge them into 1 csv merged.csv 1,2 2,3 3,5 4,2
Swargam Sagarika (4 months ago)
Don't we have to mention path of csv file?
Ashish Niranjan (4 months ago)
very nice video. i need one help on parsing csv file from s3 bucket. could you please tell me how ?
pardeep kumar (4 months ago)
is this python 2 or 3? i am working on python 3 and i see some odd behavior with reading tab delimited file
Joe Tang (4 months ago)
very practical tricks
Trying2 Understand (5 months ago)
I really enjoyed this tutorial as well as all of your other tutorials that I have seen. I was wondering if you had shown a method for updating csv files?
Anthony Dyer (5 months ago)
A message to those making python files. Don't name your file csv.py. It will not work. I fell for that. Also you need to add the argument newline='' after 'w' or your new csv file will write to every other line. That is single quotes with no space in between.
LALIT BHAVSAR (5 months ago)
How about column how separate the also like row.. please replay
Hamza Mahmood (5 months ago)
i m giving list to csv.writerow butit is taking it as a single value and not removing brackets
Thiyagu Mani (5 months ago)
csv module not found in pip
yadi刘亚娣 (5 months ago)
Thanks for the helpful video! I'm new to Python... here in the video I don't understand why the output is not a dictionary format with DictWriter. Can anyone explain it to me please? Thanks
Pedro Rodrigues (5 months ago)
Does this works for a document .txt?
Corey Schafer (5 months ago)
Hmm, that's a good question. I would assume it would because I don't think Python pays any attention to file extensions. It's definitely worth testing though.
estephanie gomez (5 months ago)
Hello Corey, Thanks for the tutorials, do you have a tutorial using csv module with tkinter? I would like to edit csv files using a GUI interface in python. Regards, Estephanie
Naveed Rajput (5 months ago)
It was fantastic , i just want to learn about online data downloading , i am able to download history data from net but when i go to write in csv file it omits some record and place dot dot dot (omit data) and in last it write some lines i am confuse how to handle that data pleas help
John Liang (5 months ago)
This video is the best, I watched many videos on YouTube, only this one really help
Tan Vorn (5 months ago)
Man how to deal with Non-Ascii characters?
Kyriakos Ketikidis (6 months ago)
How can I save my csv file in a folder and make my python script open it from there? For me it works only if I have saved my csv in my Desktop
bull shit
Dan Muniz (6 months ago)
Thanks Corey. Very Helpful! Cheers!
Touay (6 months ago)
Good vid. I hope you don't mind, and I know I am being a touch pedantic here, but for your beginners who are not used to american pronunciation, it would be more clear if pronounced "writer" with a t, rather than it sounding like "rider". I am English and it gave me a moments confusion, only rectified by reading the text, so I have to think the international people who may only be listening, could be confused.
Tahajjat Begum (6 months ago)
can u tell me how we can do customer segmentation using jypether
Ayushi Dutt (6 months ago)
where is the csv file saved ?
Kirup Vela (6 months ago)
You covered very well through explaining simple way. Thank you very much
_SealedKiller_ (7 months ago)
You can parse this without any module too
Erkam TOKGÖZ (7 months ago)
How do you put "#" on more than one line at the same time? (09:38) Thank you for your answer.
thelastcat (2 months ago)
ctrl + /
Hoàng Lực (7 months ago)
May I ask you a question? When I use csv.writerow I alway have a blank row after each row what have data, can you help me to fix it?
Prashant kumar (7 months ago)
really helpful video thank you!!
Gabe Braden (8 months ago)
Hey Corey! I was wondering how you get your csv file to "talk" to your python code. Whenever I try to run my code in my terminal, it gives me an error saying that "Traceback (most recent call last): File "csv_to_python.py", line 5, in <module> with open("app.csv", "r") as f: FileNotFoundError: [Errno 2] No such file or directory: 'app.csv'" Thank you so much!
Corey Schafer (8 months ago)
Are you sure you have your CSV in the same directory as the Python script you're running? If it's not in the same folder then it won't know how to find it. If you want to have the CSV file in a different location then you will need to pass in the full path to the CSV into the open function.
Andri Hidayat (8 months ago)
I already searching for this tutorial for some years, but finally I found it here. I don't know if I didn't found your video maybe my life is more difficult. Hope you continue in making very good quality video like this. I must subscribe your channel . Thank you so much.
tamil stalin (8 months ago)
Hi Corey, If possible , can explain how to use TextFSM module to parse collection of structure data to CSV format
Ajay R (8 months ago)
Hey Corey thanks for the awesome video . I have a question how do we get a user input and write that date to a particular row or coloum in csv file. I have a csv file which has name and team as feilds and some data in it. I would like to get user input and update my data row or coloum on it. Your help is appreciated. Pls feel free to contact me on my tweeter account as well @ajay_r_aj
H. khezri (8 months ago)
Hi I am writing nptconfig file I calculate dimensions fixed all not backbone set 300k as well as "p2npt_fixed.pdb" created now for writing a config file, I faced with this error atomsel: writepdb failed. atomsel: writepdb failed. while executing "$all writepdb ${basename}_fixed.pdb" (procedure "write_files" line 64) invoked from within "write_files" (procedure "::NAMDgui::config_only" line 3) invoked from within "::NAMDgui::config_only " (procedure "::NAMDgui::opendialog" line 96) invoked from within "::NAMDgui::opendialog save $::NAMDgui::namdconfig" invoked from within ".namdgui.go.write invoke" ("uplevel" body line 1) invoked from within "uplevel #0 [list $w invoke]" (procedure "tk::ButtonUp" line 24) invoked from within "tk::ButtonUp .namdgui.go.write" (command bound to event) can you help me?
Gene Yuss (8 months ago)
I LOVE YOU!
David Alexander (8 months ago)
I'm running 3.6 and threw an error, TypeError: "delimiter" must be a 1-character string. Solution: swapped and '/t' with ','
alhurayas m (8 months ago)
thank you good tutorial but writerow
Dev Slaven (9 months ago)
Any time I'm hunting for information on Python, my searches always lead to you and I'm never disappointed. Keep up the great work!
sabarish amaravadi (9 months ago)
i am trying to read a csv file which looks like this ['261', '92'] ['276', '92'] ['276', '117'] ['261', '117'] and i have 2 strings in a row and 400+ lines. how can i convert this to array(x, 4 ) array or list or tuple and i also wanted to remove the repeated values. For ex. here i need 261,92,276,117 in one row.
Andrew Cekuta (9 months ago)
great video once again
Prabhakaran Subramaniyan (9 months ago)
its very nice video and with many information and tips:)
Rocky Xiao (9 months ago)
We could use a "csv_writer.writerows(csv_reader)" instead of a for loop of csv_writer.writerow(line).
Vish (9 months ago)
Nice video. How can I write headers and rows for nested dictionaries? E.g. full_dict = {first_dict:{'a':1, 'b':2}, second_dict:{'a':1, 'b':2}}
omkar khare (9 months ago)
Very well explained
Songtao (10 months ago)
Awesome video!
Hi! I wrote your code is sublime text 3 and when I tried to create new file 'new_names1.csv' with a tab as a delimiter I got empty lines after each name. As I see, there are no empty lines in your video. Why could this happen? Here's a screen: https://ibb.co/fotZby
Jonathan Pereira (10 months ago)
Great!!! \o/
Hi, Jonathan. Thank you very much for the link, it helped to solve the issue! Now the code works without creating empty lines
Jonathan Pereira (10 months ago)
I've found this as well, maybe could fix that issue: https://stackoverflow.com/a/3348664/9968674
Jonathan Pereira (10 months ago)
And send me details as OS, Python Version, etc...
Jonathan Pereira (10 months ago)
Hi Рустам, it's really intriguing me! Could you send me your whole code (zip)? [email protected] Then I'll try to figure it out here... (Just if you want...)
Rithesh B Canvas (10 months ago)
Super .. helpfull.. Please do this kind of real word problems..
Swapnil Sawant (10 months ago)
Great
Bilal Khan (10 months ago)
Great! Your tutorials are awesome
André Neiva Liboreiro (10 months ago)
Hi Corey, I am runnig the same code as you did to get the info from Beatles albums on my example. The thing is that when I run this on repl.it environment, it works fine (see the link https://repl.it/@AndreNeiva/PleasedWarmheartedExperiment). However, when I run the same script on my GIT BASH Python I get unicoded values ('—' or '\xe2\x80\x94') for some keys (see the link here https://repl.it/@AndreNeiva/MeaslyInsistentArea ). PS: On my PC, I'm running a 3.6.4 Python - Anaconda w/ GIT BASH and I get the same trouble with the unicodes. I tried basically every decode and encode method combinations but none solve the problem.
Ivan Stepanoff (10 months ago)
And separate thanks for clear speaking!!! I understand all the words. Beautiful!
Ivan Stepanoff (10 months ago)
Can't be better.. And "ordered dictionaries" is my new discovery. )) I knew that method to constract dict, but didn't recognize, that the result was odered.
Valerie Wang (10 months ago)
OMG Corey you are soooooo helpfullllllllllll. I love you!
Jingyu Chang (11 months ago)
I just realized whenever I have a question about python, as long as I come to your channel, I can find the answer! Thanks very much!
Mahi R (11 months ago)
I am using Standard Python IDLE and for writing new csv it is showing as "FileNotFoundError: [Errno 2] No such file or directory: 'new_csv.csv'"
Mayur Kr. Garg (5 months ago)
Make sure you are passing the 'w' argument while writing.
Ross Crispin (11 months ago)
Hey Corey, thanks for your videos. I was wondering, how would I update a CSV file that has been created already? I'm currently working on a news website web scraping tool that parses the words in each headline. The idea is to create a content analysis program, so ideally I need the CSV file to update rather than overwrite each time it's executed. How would I go about this? Kind regards, Ross
Nick Larralde (11 months ago)
I know this is an older video so hopefully, someone is still looking at comments. When you did the "With Line in csv_reader:" how did it naturally know that the variable "line" should be a single line from the .csv file?
Nick Larralde (11 months ago)
Ah, thank you Corey. That makes sense. Am I correct to assume that for different iterables, the variable assignment could be different? In this case, a for <variable> assumes that the <variable> is separated by the line whereas a different object could behave differently?
Corey Schafer (11 months ago)
It's something I've looked up in the documentation. Python has objects that are "iterable", and when you loop over them they can return different objects. In this case, the csv_reader returns those lines.

Would you like to comment?

Join YouTube for a free account, or sign in if you are already a member.