import pandas as pd
data = {'city': ['Nagano', 'Sydney', 'Salt Lake City', 'Athens', 'Torino', 'Beijing', 'Vancouver', 'London', 'Sochi', 'Rio de Janeiro'],
'year': [1998, 2000, 2002, 2004, 2006, 2008, 2010, 2012, 2014, 2016],
'season': ['winter', 'summer', 'winter', 'summer', 'winter', 'summer', 'winter', 'summer', 'winter', 'summer']}
df = pd.DataFrame(data)
df.to_csv("csv.csv")
これを実行するとcsv.csvというCSVデータが同じディレクトリに作成
|
1 2 3 4 5 6 7 |
# データセット読込 from pandas import read_csv dataframe = read_csv('aaa.csv', usecols=[1],engine='python', skipfooter=3) dataset = dataframe.values dataset = dataset.astype('float32') |
usecols=[1]は読み込むcsvの列を指定[1]であれば２列目を縦に読み込む
engine=’python’ か’c’のどちらか
skipfooter=3はデータ末尾の3行はフッターとみなし読み込まない
|
1 2 |
#カラム（一行目にそれぞれの数値が何を表しているか） dataset.columns=["sepal length", "sepal width", "petal length", "petal width", "class"] |
わかりやすい一覧↓
Pandasのread_csvの全引数を解説
DataFrame型の変数に対して、変数.diff(変数.diff - x, axis="0 or 1")と指定で
行間または列間の差を計算したDataFrameが作成
第1引数が正の場合は前の行との差、負の場合は後の行との差
axisは0の場合が行(横)方向、1の場合が列(縦)方向です。
|
1 |
<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">as</span> <span class="cm-variable">np</span></span> |
|
1 |
<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span> |
|
1 |
<span role="presentation"><span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">seed</span>(<span class="cm-number">0</span>)</span> |
|
1 |
<span role="presentation"><span class="cm-variable">columns</span> = [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"kiwifruit"</span>]</span> |
|
1 |
<span role="presentation"></span> |
|
1 |
<span role="presentation"><span class="cm-comment"># DataFrameを生成し、列を追加</span></span> |
|
1 |
<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>()</span> |
|
1 |
<span role="presentation"><span class="cm-keyword">for</span> <span class="cm-variable">column</span> <span class="cm-keyword">in</span> <span class="cm-variable">columns</span>:</span> |
|
1 |
<span role="presentation"> <span class="cm-variable">df</span>[<span class="cm-variable">column</span>] = <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">choice</span>(<span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>), <span class="cm-number">10</span>)</span> |
|
1 |
<span role="presentation"><span class="cm-variable">df</span>.<span class="cm-property">index</span> = <span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>)</span> |
|
1 |
<span role="presentation"></span> |
|
1 |
<span role="presentation"><span class="cm-comment"># dfの各行について、2行後の行との差を計算したDataFrameをdf_diffに代入</span></span> |
|
1 2 3 |
<span role="presentation"><span class="cm-variable">df_diff</span>=<span class="cm-variable">df</span>.<span class="cm-property">diff</span> <span class=" CodeMirror-matchingbracket">(</span><span class="cm-operator">-</span><span class="cm-number">2</span>,<span class="cm-variable">axis</span> = <span class="cm-number">0</span><span class=" CodeMirror-matchingbracket">) #第一引数がマイナスの場合は下、右方向へ処理 第一引数がプラスの場合は上、左方向へ処理</span></span> |
|
1 |
</code><code> |
|
1 |
<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df</span>)</span> |
|
1 |
<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df_diff</span>)</span> |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
apple orange banana strawberry kiwifruit 1 6 8 6 3 10 2 1 7 10 4 10 3 4 9 9 9 1 4 4 9 10 2 5 5 8 2 5 4 8 6 10 7 4 4 4 7 4 8 1 4 3 8 6 8 4 8 8 9 3 9 6 1 3 10 5 2 1 2 1 apple orange banana strawberry kiwifruit 1 2.0 -1.0 -3.0 -6.0 9.0 2 -3.0 -2.0 0.0 2.0 5.0 3 -4.0 7.0 4.0 5.0 -7.0 4 -6.0 2.0 6.0 -2.0 1.0 5 4.0 -6.0 4.0 0.0 5.0 6 4.0 -1.0 0.0 -4.0 -4.0 7 1.0 -1.0 -5.0 3.0 0.0 8 1.0 6.0 3.0 6.0 7.0 9 NaN NaN NaN NaN NaN 10 NaN NaN NaN NaN NaN |
列ごとの平均値、最大値、最小値等統計的情報を要約統計量と呼ぶ。
DataFrameの変数に対して、変数.describe()は変数の列ごとの
個数、平均値、標準偏差、最小値、四分位数、最大値を返す。
DataFrameのインデックスの数字は統計量の名前に置き換わる
|
1 |
<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">as</span> <span class="cm-variable">np</span></span> |
|
1 |
<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span> |
|
1 |
<span role="presentation"><span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">seed</span>(<span class="cm-number">0</span>)</span> |
|
1 |
<span role="presentation"><span class="cm-variable">columns</span> = [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"kiwifruit"</span>]</span> |
|
1 |
<span role="presentation"></span> |
|
1 |
<span role="presentation"><span class="cm-comment"># DataFrameを生成し、列を追加</span></span> |
|
1 |
<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>()</span> |
|
1 |
<span role="presentation"><span class="cm-keyword">for</span> <span class="cm-variable">column</span> <span class="cm-keyword">in</span> <span class="cm-variable">columns</span>:</span> |
|
1 |
<span role="presentation"> <span class="cm-variable">df</span>[<span class="cm-variable">column</span>] = <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">choice</span>(<span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>), <span class="cm-number">10</span>)</span> |
|
1 |
<span role="presentation"><span class="cm-variable">df</span>.<span class="cm-property">index</span> = <span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>)</span> |
|
1 |
<span role="presentation"><span class="cm-builtin">print</span><span class=" CodeMirror-matchingbracket">(</span><span class="cm-variable">df</span><span class=" CodeMirror-matchingbracket">)</span></span> |
|
1 |
<span role="presentation"><span class="cm-comment"># dfの要約統計量のうち、"mean", "max", "min"を取り出してdf_desに代入してください</span></span> |
|
1 |
<span role="presentation"></span> |
|
1 |
<span role="presentation"><span class="cm-variable">df_des</span> = <span class="cm-variable">df</span>.<span class="cm-property">describe</span>().<span class="cm-property">loc</span>[[<span class="cm-string">"mean"</span>, <span class="cm-string">"max"</span>, <span class="cm-string">"min"</span>]]</span> |
|
1 |
<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df_des</span>)</span> |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
apple orange banana strawberry kiwifruit 1 6 8 6 3 10 2 1 7 10 4 10 3 4 9 9 9 1 4 4 9 10 2 5 5 8 2 5 4 8 6 10 7 4 4 4 7 4 8 1 4 3 8 6 8 4 8 8 9 3 9 6 1 3 10 5 2 1 2 1 ↓ apple orange banana strawberry kiwifruit mean 5.1 6.9 5.6 4.1 5.3 max 10.0 9.0 10.0 9.0 10.0 min 1.0 2.0 1.0 1.0 1.0 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
import pandas as pd # 注文情報 order_df = pd.DataFrame([[1000, 2546, 103], [1001, 4352, 101], [1002, 342, 101]], columns=["id", "item_id", "customer_id"]) # 顧客情報 customer_df = pd.DataFrame([[101, "Tanaka"], [102, "Suzuki"], [103, "Kato"]], columns=["id", "name"]) # customer_dfを元に"name"をorder_dfに結合してorder_dfに代入してください order_df = pd.merge(order_df,customer_df,left_on="customer_id",right_on="id",how="inner") print(order_df) |
|
1 2 3 4 |
id_x item_id customer_id id_y name 0 1000 2546 103 103 Kato 1 1001 4352 101 101 Tanaka 2 1002 342 101 101 Tanaka |
DataFrameの結合で共通するものが無くても結合し、
数値の孫座しない部分はNoneとなる
|
1 2 3 |
<span role="presentation"><span class="cm-keyword"> import</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">as</span> <span class="cm-variable">np</span></span> |
|
1 |
<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span> |
|
1 |
<span role="presentation"></span> |
|
1 |
<span role="presentation"><span class="cm-variable">data1</span> = {<span class="cm-string">"fruits"</span>: [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"kiwifruit"</span>],</span> |
|
1 |
<span role="presentation"> <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2006</span>],</span> |
|
1 |
<span role="presentation"> <span class="cm-string">"amount"</span>: [<span class="cm-number">1</span>, <span class="cm-number">4</span>, <span class="cm-number">5</span>, <span class="cm-number">6</span>, <span class="cm-number">3</span>]}</span> |
|
1 |
<span role="presentation"><span class="cm-variable">df1</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data1</span>)</span> |
|
1 |
<span role="presentation"></span> |
|
1 |
<span role="presentation"><span class="cm-variable">data2</span> = {<span class="cm-string">"fruits"</span>: [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"mango"</span>],</span> |
|
1 |
<span role="presentation"> <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2007</span>],</span> |
|
1 |
<span role="presentation"> <span class="cm-string">"price"</span>: [<span class="cm-number">150</span>, <span class="cm-number">120</span>, <span class="cm-number">100</span>, <span class="cm-number">250</span>, <span class="cm-number">3000</span>]}</span> |
|
1 |
<span role="presentation"><span class="cm-variable">df2</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data2</span>)</span> |
|
1 |
<span role="presentation"></span> |
|
1 |
<span role="presentation"><span class="cm-comment"># df1, df2の中身を確認してください</span></span> |
|
1 |
<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df1</span>)</span> |
|
1 |
<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df2</span>)</span> |
|
1 |
<span role="presentation"></span> |
|
1 |
<span role="presentation"><span class="cm-comment"># df1とdf2を"fruits"をキーに外部結合して作成したDataFrameをdf3に代入してください</span></span> |
|
1 |
<span role="presentation"><span class="cm-variable">df3</span> = <span class="cm-variable">pd</span>.<span class="cm-property">merge</span>(<span class="cm-variable">df1</span>, <span class="cm-variable">df2</span>, <span class="cm-variable">on</span>=<span class="cm-string">"fruits"</span>, <span class="cm-variable">how</span>=<span class="cm-string">"outer"</span>)</span> |
|
1 |
<span role="presentation"></span> |
|
1 |
<span role="presentation"><span class="cm-builtin">print</span><span class=" CodeMirror-matchingbracket">(</span><span class="cm-variable">df3</span><span class=" CodeMirror-matchingbracket">)</span></span> |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
amount fruits year 0 1 apple 2001 1 4 orange 2002 2 5 banana 2001 3 6 strawberry 2008 4 3 kiwifruit 2006 fruits price year 0 apple 150 2001 1 orange 120 2002 2 banana 100 2001 3 strawberry 250 2008 4 mango 3000 2007 amount fruits year_x price year_y 0 1.0 apple 2001.0 150.0 2001.0 1 4.0 orange 2002.0 120.0 2002.0 2 5.0 banana 2001.0 100.0 2001.0 3 6.0 strawberry 2008.0 250.0 2008.0 4 3.0 kiwifruit 2006.0 NaN NaN 5 NaN mango NaN 3000.0 2007.0 |