import pandas as pd
df = pd.read_csv(“http://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data”, header=None)
df.columns=[“”, “Alcohol”, “Malic acid”, “Ash”, “Alcalinity of ash”, “Magnesium”,”Total phenols”, “Flavanoids”, “Nonflavanoid phenols”, “Proanthocyanins”,”Color intensity”, “Hue”, “OD280/OD315 of diluted wines”, “Proline”]
df[“Nonflavanoid phenols”].mean()
|
1 2 |
|
1 2 3 4 5 6 7 8 9 10 |
import numpy as np from numpy import nan as NA import pandas as pd aa = pd.DataFrame(np.random.rand(10,4)) #一部のデータNanにする sample_data_frame.iloc[1,0] = NA sample_data_frame.iloc[2,2] = NA sample_data_frame.iloc[5:,3] = NA |
fillna()を用いると、引数として与えた数をNaNの部分に代入、
この例では0を代入
|
1 2 |
sample_data_frame.fillna(0) |
methodをにffillを指定し、上の値がそのままコピーされる
|
1 2 |
<span class="cm-variable">sample_data_frame</span>.<span class="cm-property">fillna</span>(<span class="cm-variable">method</span>=<span class="cm-string">"ffill"</span>) |
|
1 2 3 4 5 |
#欠損値処理 df['Fare'] = df['Fare'].fillna(df['Fare'].median())#Fareの平均値を代入 df['Age'] = df['Age'].fillna(df['Age'].median())#Ageの平均値を代入 df['Embarked'] = df['Embarked'].fillna('S')#Embarkedの値はQ,O,Sのどれかなので代表的なSを代入 df=df.dropna(subset=['Age'])#AgeのカラムにNanがある物を削除 |
|
1 2 3 4 5 6 7 8 9 10 11 12 |
<span class="cm-keyword">import</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">as</span> <span class="cm-variable">np</span> <span class="cm-keyword">from</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">import</span> <span class="cm-variable">nan</span> <span class="cm-keyword">as</span> <span class="cm-variable">NA</span> <span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span> <span class="cm-variable">sample_data_frame</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">rand</span>(<span class="cm-number">10</span>,<span class="cm-number">4</span>)) <span class="cm-comment">#一部のデータをわざと欠損</span> <span class="cm-variable">sample_data_frame</span>.<span class="cm-property">iloc</span>[<span class="cm-number">1</span>,<span class="cm-number">0</span>] = <span class="cm-variable">NA</span> <span class="cm-variable">sample_data_frame</span> |
dropnaを用いると、NaNのある行を全て取り除く
|
1 2 |
<span class="cm-variable">sample_data_frame</span>.<span class="cm-property">dropna</span>() |
ペアワイズ削除
|
1 2 3 |
<span class="cm-variable">sample_data_frame</span>[[<span class="cm-number">0</span>,<span class="cm-number">1</span>]].<span class="cm-property">dropna</span>() </code><code class="cm-s-ipython language-python"> |
|
1 |
|
1 |
print(1)import pandas as pd
data = {'city': ['Nagano', 'Sydney', 'Salt Lake City', 'Athens', 'Torino', 'Beijing', 'Vancouver', 'London', 'Sochi', 'Rio de Janeiro'],
'year': [1998, 2000, 2002, 2004, 2006, 2008, 2010, 2012, 2014, 2016],
'season': ['winter', 'summer', 'winter', 'summer', 'winter', 'summer', 'winter', 'summer', 'winter', 'summer']}
df = pd.DataFrame(data)
df.to_csv("csv.csv")
これを実行するとcsv.csvというCSVデータが同じディレクトリに作成
import csv
import time
from datetime import datetime
with open(‘csv0.csv’, ‘w’) as csvfile:
writer = csv.writer(csvfile, lineterminator=’\n’)
writer.writerow([‘2’, ‘3’, ‘5’])
writer.writerow([‘7′, ’11’, ’13’])
writer.writerow([datetime.now(),value])#関数、変数もカンマ区切りで入る
csv0.csvというファイルが同じディレクトリに生成される