Rate this:
I am trying to read the following file and store into an array. First I need to split the strings sep Further more i need to plot it. Can some one help me please?
:21:49,7 ;00;00;00;;10;E008
:21:49,8 ;00;00;00;;10;E008
:21:49,9 ;00;00;00;;10;E008
:21:49,0 ;00;00;00;;10;E008
:21:50,1 ;00;00;00;;10;E008
:21:50,2 ;00;00;00;;10;E008
:21:50,3 ;00;00;00;;10;E008
:21:50,4 ;00;00;00;;10;E008
:21:50,5 ;00;00;00;;10;E008
:21:50,6 ;00;00;00;;10;E008
:21:50,7 ;00;00;00;;10;E008
:21:50,8 ;00;00;00;;10;E008
What I have tried:
I have tried to read the file and store it in an array
using System.Collections.G
using System.IO;
using System.T
namespace ComPlay
class Read
private string[]
private float[,]
private int nL
private int nC
public Read()
public int get_nLines()
public int get_nColumns()
public float[,] get_Data()
public string[] get_Header()
public Read(Stream myStream)
StreamReader sr = new StreamReader(myStream);
aux = sr.ReadLine();
header = aux.Split(' ');
nColumns = header.L
nLines = 0;
while ((aux = sr.ReadLine()) != null)
if (aux.Length & 0) nLines++;
data = new float[nLines, nColumns];
sr.BaseStream.Seek(0, 0);
sr.ReadLine();
for (int i = 0; i & nL i++)
aux = sr.ReadLine();
{ pieces = aux.Split(' '); }
{ pieces = aux.Split(';');}
for (int j = 0; j & nC j++)
data[i, j] = float.Parse(pieces[j]);
sr.Close();
Posted 2-Mar-16 3:55am
Updated 3-Mar-16 0:27am
Solution 3
Have you tried regular expressions?
It?s very useful to know how they work (painful experience).
var table = new List&string[]&();
using (var r = new StreamReader("filePathOrStreamHere"))
while (!r.EndOfStream)
string line = r.ReadLine();
table.Add(Regex.Split(line, @"\s|[;]|[,]"));
r.Close();
When you are using the data later you can convert it into the datatype based on index?
Posted 2-Mar-16 20:30pm
Solution 1
As Sascha says, a pre-built CSV reader is a much better idea - and the one he suggests is the one I use as well: []
For charting, it's pretty easy once you've read the data: [] should help.
Posted 2-Mar-16 4:15am
Solution 2
Why to force doors wide open?
Use []. It is a set of classes that expose data access services to the .NET programmer.
Posted 2-Mar-16 7:33am
<button class="toolbar" onclick="editor_ctl00_ctl00_MC_AMC_PostEntryObj_Content.insertText('&l' + 't;');" title="Insert HTML Extended Character Code For &&&
Strip HTML
Encode HTML
Paste as-is
Code block
Quoted Text
Best guess
Treat my content as plain text, not as HTML
Your Email &
Password &
Your Email &
email is in use. Do you need your ?
Optional Password &
I have read and agree to the
Please subscribe me to the CodeProject newsletters
When answering a question please:
Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad
spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or
edit the question and fix the problem. Insults are not welcome.
Let's work to help developers, not make them feel stupid.
This content, along with any associated source code and files, is licensed under
Last Updated 3 Mar 2016
Copyright & ,
All Rights Reserved.
CodeProject,
503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada
+1 416-849-8900 x 100reading csv files in scipy/numpy in Python - Stack Overflow
to customize your list.
Join the Stack Overflow Community
Stack Overflow is a community of 6.3 million programmers, just like you, helping each other.
J it only takes a minute:
I am having trouble reading a csv file, delimited by tabs, in python. I use the following function:
def csv2array(filename, skiprows=0, delimiter='\t', raw_header=False, missing=None, with_header=True):
Parse a file name into an array. Return the array and additional header lines. By default,
parse the header lines into dictionaries, assuming the parameters are numeric,
using 'parse_header'.
f = open(filename, 'r')
skipped_rows = []
for n in range(skiprows):
header_line = f.readline().strip()
if raw_header:
skipped_rows.append(header_line)
skipped_rows.append(parse_header(header_line))
if missing:
data = genfromtxt(filename, dtype=None, names=with_header,
deletechars='', skiprows=skiprows, missing=missing)
if delimiter != '\t':
data = genfromtxt(filename, dtype=None, names=with_header, delimiter=delimiter,
deletechars='', skiprows=skiprows)
data = genfromtxt(filename, dtype=None, names=with_header,
deletechars='', skiprows=skiprows)
if data.ndim == 0:
data = array([data.item()])
return (data, skipped_rows)
the problem is that genfromtxt complains about my files, e.g. with the error:
Line #27100 (got 12 columns instead of 16)
I am not sure where these errors come from. Any ideas?
Here's an example file that causes the problem:
C-1 C-2 C-5 genesymbol
guanine nucleotide binding protein alpha
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Pbsn
Is there a better way to write a generic csv2array function?
14.1k70211329
Check out the python CSV module:
import csv
reader = csv.reader(open("myfile.csv", "rb"),
delimiter='\t', quoting=csv.QUOTE_NONE)
header = []
records = []
fields = 16
if thereIsAHeader: header = reader.next()
for row, record in enumerate(reader):
if len(record) != fields:
print "Skipping malformed record %i, contains %i fields (%i expected)" %
(record, len(record), fields)
records.append(record)
# do numpy stuff.
8,81924079
May I ask why you're not using the built-in csv reader?
I've used it very effectively with numpy/scipy. I would share my code but unfortunately it's owned by my employer, but it should be very straightforward to write your own.
59.9k37177283
Likely it came from Line 27100 in your data file... and it had 12 columns instead of 16. I.e. it had:
separator,1,2,3,4,5,6,7,8,9,10,11,12,separator
And it was expecting something like this:
separator,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,separator
I'm not sure how you want to convert your data, but if you have irregular line lengths, the easiest way would be something like this:
lines = f.read().split('someseparator')
for line in lines:
splitline = line.split(',')
#do something with splitline
19.2k872139
I have successfully us (1): if I simply need to read arbitrary CSV, I used the CSV module (as pointed out by other users), and (2): if I require repeated processing of a known CSV (or any) format, I write a simple parser.
It seems that your problem fits in the second category, and a parser should be very simple:
f = open('file.txt', 'r').readlines()
for line in f:
tokens = line.strip().split('\t')
gene = tokens[0]
vals = [float(k) for k in tokens[1:10]]
stuff = tokens[10:]
# do something with gene, vals, and stuff
You can add a line in the reader for skipping comments (`if tokens[0] == '#': continue') or to handle blank lines ('if tokens == []: continue'). You get the idea.
19.1k1163105
I think Nick T's approach would be the better way to go. I would make one change. As I would replace the following code:
for row, record in enumerate(reader):
if len(record) != fields:
print "Skipping malformed record %i, contains %i fields (%i expected)" %
(record, len(record), fields)
records.append(record)
records = np.asrray([row for row in reader if len(row) = fields ])
print('Number of skipped records: %i'%(len(reader)-len(records)) #note you have to do more than len(reader) as an iterator does not have a length like a list or tuple
The list comprehension will return a numpy array and take advantage of pre-compiled libraries which should speed things up greatly. Also, I would recommend using print() as a function versus print "" as the former is the standard for python3 which is most likely the future and I would use
over print.
Your Answer
Sign up or
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Post as a guest
By posting your answer, you agree to the
Not the answer you&#39;re looking for?
Browse other questions tagged
Stack Overflow works best with JavaScript enabled