プログラミング

DataFrame 要約計量　平均、最大、最小、標準偏差等

2017年11月26日 by 河副太智 Leave a Comment

列ごとの平均値、最大値、最小値等統計的情報を要約統計量と呼ぶ。
DataFrameの変数に対して、変数.describe()は変数の列ごとの
個数、平均値、標準偏差、最小値、四分位数、最大値を返す。

DataFrameのインデックスの数字は統計量の名前に置き換わる

<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">as</span> <span class="cm-variable">np</span></span>

				1

						<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">as</span> <span class="cm-variable">np</span></span>

<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span>

				1

						<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span>

<span role="presentation"><span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">seed</span>(<span class="cm-number">0</span>)</span>

				1

						<span role="presentation"><span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">seed</span>(<span class="cm-number">0</span>)</span>

<span role="presentation"><span class="cm-variable">columns</span> = [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"kiwifruit"</span>]</span>

				1

						<span role="presentation"><span class="cm-variable">columns</span> = [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"kiwifruit"</span>]</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-comment"># DataFrameを生成し、列を追加</span></span>

				1

						<span role="presentation"><span class="cm-comment"># DataFrameを生成し、列を追加</span></span>

<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>()</span>

				1

						<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>()</span>

<span role="presentation"><span class="cm-keyword">for</span> <span class="cm-variable">column</span> <span class="cm-keyword">in</span> <span class="cm-variable">columns</span>:</span>

				1

						<span role="presentation"><span class="cm-keyword">for</span> <span class="cm-variable">column</span> <span class="cm-keyword">in</span> <span class="cm-variable">columns</span>:</span>

<span role="presentation">    <span class="cm-variable">df</span>[<span class="cm-variable">column</span>] = <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">choice</span>(<span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>), <span class="cm-number">10</span>)</span>

				1

						<span role="presentation">    <span class="cm-variable">df</span>[<span class="cm-variable">column</span>] = <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">choice</span>(<span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>), <span class="cm-number">10</span>)</span>

<span role="presentation"><span class="cm-variable">df</span>.<span class="cm-property">index</span> = <span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>)</span>

				1

						<span role="presentation"><span class="cm-variable">df</span>.<span class="cm-property">index</span> = <span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>)</span>

<span role="presentation"><span class="cm-builtin">print</span><span class=" CodeMirror-matchingbracket">(</span><span class="cm-variable">df</span><span class=" CodeMirror-matchingbracket">)</span></span>

				1

						<span role="presentation"><span class="cm-builtin">print</span><span class=" CodeMirror-matchingbracket">(</span><span class="cm-variable">df</span><span class=" CodeMirror-matchingbracket">)</span></span>

<span role="presentation"><span class="cm-comment"># dfの要約統計量のうち、"mean", "max", "min"を取り出してdf_desに代入してください</span></span>

				1

						<span role="presentation"><span class="cm-comment"># dfの要約統計量のうち、"mean", "max", "min"を取り出してdf_desに代入してください</span></span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-variable">df_des</span> = <span class="cm-variable">df</span>.<span class="cm-property">describe</span>().<span class="cm-property">loc</span>[[<span class="cm-string">"mean"</span>, <span class="cm-string">"max"</span>, <span class="cm-string">"min"</span>]]</span>

				1

						<span role="presentation"><span class="cm-variable">df_des</span> = <span class="cm-variable">df</span>.<span class="cm-property">describe</span>().<span class="cm-property">loc</span>[[<span class="cm-string">"mean"</span>, <span class="cm-string">"max"</span>, <span class="cm-string">"min"</span>]]</span>

<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df_des</span>)</span>

				1

						<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df_des</span>)</span>

    apple  orange  banana  strawberry  kiwifruit
1       6       8       6           3         10
2       1       7      10           4         10
3       4       9       9           9          1
4       4       9      10           2          5
5       8       2       5           4          8
6      10       7       4           4          4
7       4       8       1           4          3
8       6       8       4           8          8
9       3       9       6           1          3
10      5       2       1           2          1
↓
      apple  orange  banana  strawberry  kiwifruit
mean    5.1     6.9     5.6         4.1        5.3
max    10.0     9.0    10.0         9.0       10.0
min     1.0     2.0     1.0         1.0        1.0

apple orange banana strawberry kiwifruit

1 6 8 6 3 10

2 1 7 10 4 10

3 4 9 9 9 1

4 4 9 10 2 5

5 8 2 5 4 8

6 10 7 4 4 4

7 4 8 1 4 3

8 6 8 4 8 8

9 3 9 6 1 3

10 5 2 1 2 1

↓

apple orange banana strawberry kiwifruit

mean 5.1 6.9 5.6 4.1 5.3

max 10.0 9.0 10.0 9.0 10.0

min 1.0 2.0 1.0 1.0 1.0

DataFrame カラムの名称が異なるが意味が同じデータを繋げる

2017年11月26日 by 河副太智 Leave a Comment

pandas.merge(左側DF, 右側DF, left_on=”左側DFのカラム”,

right_on=”右側DFのカラム”, how=”結合方法”)

import pandas as pd

# 注文情報
order_df = pd.DataFrame([[1000, 2546, 103],
                         [1001, 4352, 101],
                         [1002, 342, 101]],
                         columns=["id", "item_id", "customer_id"])
# 顧客情報
customer_df = pd.DataFrame([[101, "Tanaka"],
                           [102, "Suzuki"],
                           [103, "Kato"]],
                           columns=["id", "name"])

# customer_dfを元に"name"をorder_dfに結合してorder_dfに代入してください
order_df = pd.merge(order_df,customer_df,left_on="customer_id",right_on="id",how="inner")

print(order_df)

import pandas as pd

# 注文情報

order_df = pd.DataFrame([[1000, 2546, 103],

[1001, 4352, 101],

[1002, 342, 101]],

columns=["id", "item_id", "customer_id"])

# 顧客情報

customer_df = pd.DataFrame([[101, "Tanaka"],

[102, "Suzuki"],

[103, "Kato"]],

columns=["id", "name"])

# customer_dfを元に"name"をorder_dfに結合してorder_dfに代入してください

order_df = pd.merge(order_df,customer_df,left_on="customer_id",right_on="id",how="inner")

print(order_df)

   id_x  item_id  customer_id  id_y    name
0  1000     2546          103   103    Kato
1  1001     4352          101   101  Tanaka
2  1002      342          101   101  Tanaka

id_x item_id customer_id id_y name

0 1000 2546 103 103 Kato

1 1001 4352 101 101 Tanaka

2 1002 342 101 101 Tanaka

DataFrame　外部結合　合わないデータも結合

2017年11月26日 by 河副太智 Leave a Comment

DataFrameの結合で共通するものが無くても結合し、
数値の孫座しない部分はNoneとなる

<span role="presentation"><span class="cm-keyword">

import</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">as</span> <span class="cm-variable">np</span></span>

import numpy as np

<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span>

1	<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span>

<span role="presentation"></span>

1	<span role="presentation"></span>

<span role="presentation"><span class="cm-variable">data1</span> = {<span class="cm-string">"fruits"</span>: [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"kiwifruit"</span>],</span>

data1 = {"fruits": ["apple", "orange", "banana", "strawberry", "kiwifruit"],

<span role="presentation">        <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2006</span>],</span>

1	<span role="presentation"> <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2006</span>],</span>

<span role="presentation">        <span class="cm-string">"amount"</span>: [<span class="cm-number">1</span>, <span class="cm-number">4</span>, <span class="cm-number">5</span>, <span class="cm-number">6</span>, <span class="cm-number">3</span>]}</span>

1	<span role="presentation"> <span class="cm-string">"amount"</span>: [<span class="cm-number">1</span>, <span class="cm-number">4</span>, <span class="cm-number">5</span>, <span class="cm-number">6</span>, <span class="cm-number">3</span>]}</span>

<span role="presentation"><span class="cm-variable">df1</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data1</span>)</span>

1	<span role="presentation"><span class="cm-variable">df1</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data1</span>)</span>

<span role="presentation"></span>

1	<span role="presentation"></span>

<span role="presentation"><span class="cm-variable">data2</span> = {<span class="cm-string">"fruits"</span>: [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"mango"</span>],</span>

data2 = {"fruits": ["apple", "orange", "banana", "strawberry", "mango"],

<span role="presentation">        <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2007</span>],</span>

1	<span role="presentation"> <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2007</span>],</span>

<span role="presentation">        <span class="cm-string">"price"</span>: [<span class="cm-number">150</span>, <span class="cm-number">120</span>, <span class="cm-number">100</span>, <span class="cm-number">250</span>, <span class="cm-number">3000</span>]}</span>

1	<span role="presentation"> <span class="cm-string">"price"</span>: [<span class="cm-number">150</span>, <span class="cm-number">120</span>, <span class="cm-number">100</span>, <span class="cm-number">250</span>, <span class="cm-number">3000</span>]}</span>

<span role="presentation"><span class="cm-variable">df2</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data2</span>)</span>

1	<span role="presentation"><span class="cm-variable">df2</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data2</span>)</span>

<span role="presentation"></span>

1	<span role="presentation"></span>

<span role="presentation"><span class="cm-comment"># df1, df2の中身を確認してください</span></span>

1	<span role="presentation"><span class="cm-comment"># df1, df2の中身を確認してください</span></span>

<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df1</span>)</span>

1	<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df1</span>)</span>

<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df2</span>)</span>

1	<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df2</span>)</span>

<span role="presentation"></span>

1	<span role="presentation"></span>

<span role="presentation"><span class="cm-comment"># df1とdf2を"fruits"をキーに外部結合して作成したDataFrameをdf3に代入してください</span></span>

1	<span role="presentation"><span class="cm-comment"># df1とdf2を"fruits"をキーに外部結合して作成したDataFrameをdf3に代入してください</span></span>

<span role="presentation"><span class="cm-variable">df3</span> = <span class="cm-variable">pd</span>.<span class="cm-property">merge</span>(<span class="cm-variable">df1</span>, <span class="cm-variable">df2</span>, <span class="cm-variable">on</span>=<span class="cm-string">"fruits"</span>, <span class="cm-variable">how</span>=<span class="cm-string">"outer"</span>)</span>

df3 = pd.merge(df1, df2, on="fruits", how="outer")

<span role="presentation"></span>

1	<span role="presentation"></span>

<span role="presentation"><span class="cm-builtin">print</span><span class=" CodeMirror-matchingbracket">(</span><span class="cm-variable">df3</span><span class=" CodeMirror-matchingbracket">)</span></span>

1	<span role="presentation"><span class="cm-builtin">print</span><span class=" CodeMirror-matchingbracket">(</span><span class="cm-variable">df3</span><span class=" CodeMirror-matchingbracket">)</span></span>

   amount      fruits  year
0       1       apple  2001
1       4      orange  2002
2       5      banana  2001
3       6  strawberry  2008
4       3   kiwifruit  2006
       fruits  price  year
0       apple    150  2001
1      orange    120  2002
2      banana    100  2001
3  strawberry    250  2008
4       mango   3000  2007
   amount      fruits  year_x   price  year_y
0     1.0       apple  2001.0   150.0  2001.0
1     4.0      orange  2002.0   120.0  2002.0
2     5.0      banana  2001.0   100.0  2001.0
3     6.0  strawberry  2008.0   250.0  2008.0
4     3.0   kiwifruit  2006.0     NaN     NaN
5     NaN       mango     NaN  3000.0  2007.0

amount fruits year

0 1 apple 2001

1 4 orange 2002

2 5 banana 2001

3 6 strawberry 2008

4 3 kiwifruit 2006

fruits price year

0 apple 150 2001

1 orange 120 2002

2 banana 100 2001

3 strawberry 250 2008

4 mango 3000 2007

amount fruits year_x price year_y

0 1.0 apple 2001.0 150.0 2001.0

1 4.0 orange 2002.0 120.0 2002.0

2 5.0 banana 2001.0 100.0 2001.0

3 6.0 strawberry 2008.0 250.0 2008.0

4 3.0 kiwifruit 2006.0 NaN NaN

5 NaN mango NaN 3000.0 2007.0

DataFrameできる事

2017年11月26日 by 河副太智 Leave a Comment

内部結合とは共通するデータのみを結合し、
共通しないデータは破棄される

■データフレーム作成

<span role="presentation"></span>

1	<span role="presentation"></span>

<span role="presentation"><span class="cm-variable">data1</span> = {<span class="cm-string">"fruits"</span>: [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"kiwifruit"</span>],</span>

<span role="presentation">        <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2006</span>],</span>

1	<span role="presentation"> <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2006</span>],</span>

<span role="presentation">        <span class="cm-string">"amount"</span>: [<span class="cm-number">1</span>, <span class="cm-number">4</span>, <span class="cm-number">5</span>, <span class="cm-number">6</span>, <span class="cm-number">3</span>]}</span>

1	<span role="presentation"> <span class="cm-string">"amount"</span>: [<span class="cm-number">1</span>, <span class="cm-number">4</span>, <span class="cm-number">5</span>, <span class="cm-number">6</span>, <span class="cm-number">3</span>]}</span>

<span role="presentation"><span class="cm-variable">df1</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data1</span>)</span>

1	<span role="presentation"><span class="cm-variable">df1</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data1</span>)</span>

<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df1</span>)


</span>

print(df1)

   amount      fruits  year
0       1       apple  2001
1       4      orange  2002
2       5      banana  2001
3       6  strawberry  2008
4       3   kiwifruit  2006

amount fruits year

0 1 apple 2001

1 4 orange 2002

2 5 banana 2001

3 6 strawberry 2008

4 3 kiwifruit 2006

<span role="presentation"> </span>

1	<span role="presentation"> </span>

`■データフレーム縦方向に直接結合`

# df_data1とdf_data2を縦方向に連結しdf1に代入
df1 = pd.concat([df_data1, df_data2], axis=0)

apple orange banana
1 45 68 37
2 48 10 88
3 65 84 71
4 68 22 89
1 38 76 17
2 13 6 2
3 73 80 77
4 10 65 72

`■データフレーム横方向に直接結合`

# df_data1とdf_data2を横方向に連結しdf1に代入
df2 = pd.concat([df_data1, df_data2], axis=1)

　　apple orange banana apple orange banana
1　　　 45 　68 　　　37 　38　　 76　　 17
2　　　 48 　10 　　　88　 13 　　6 　　　2
3 　　　65 　84 　　　71 　73 　　80　　 77
4 　　　68 　22 　　　89 　10 　　65 　　72

■ソート

df = df.sort_values(カラムの変数””はなし)

■フィルタリング　条件抽出

columns = [“apple”, “orange”, “banana”, “strawberry”, “kiwifruit”]

<span role="presentation"><span class="cm-comment"># DataFrameを生成し、列を追加</span></span>

				1

						<span role="presentation"><span class="cm-comment"># DataFrameを生成し、列を追加</span></span>

<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>()</span>

				1

						<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>()</span>

<span role="presentation"><span class="cm-keyword">for</span> <span class="cm-variable">column</span> <span class="cm-keyword">in</span> <span class="cm-variable">columns</span>:</span>

				1

						<span role="presentation"><span class="cm-keyword">for</span> <span class="cm-variable">column</span> <span class="cm-keyword">in</span> <span class="cm-variable">columns</span>:</span>

<span role="presentation">    <span class="cm-variable">df</span>[<span class="cm-variable">column</span>] = <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">choice</span>(<span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>), <span class="cm-number">10</span>)</span>

				1

						<span role="presentation">    <span class="cm-variable">df</span>[<span class="cm-variable">column</span>] = <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">choice</span>(<span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>), <span class="cm-number">10</span>)</span>

<span role="presentation"><span class="cm-variable">df</span>.<span class="cm-property">index</span> = <span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>)</span>

				1

						<span role="presentation"><span class="cm-variable">df</span>.<span class="cm-property">index</span> = <span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>)</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-comment"># dfの"apple"列が5以上かつ"kiwifruit"列が5以上の値をもつ行を含むDataFrameをdfに代入してください</span></span>

				1

						<span role="presentation"><span class="cm-comment"># dfの"apple"列が5以上かつ"kiwifruit"列が5以上の値をもつ行を含むDataFrameをdfに代入してください</span></span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">df</span>.<span class="cm-property">loc</span>[<span class="cm-variable">df</span>[<span class="cm-string">"apple"</span>] <span class="cm-operator">&gt;</span>= <span class="cm-number">5</span>]</span>

				1

						<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">df</span>.<span class="cm-property">loc</span>[<span class="cm-variable">df</span>[<span class="cm-string">"apple"</span>] <span class="cm-operator">&gt;</span>= <span class="cm-number">5</span>]</span>

<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">df</span>.<span class="cm-property">loc</span>[<span class="cm-variable">df</span>[<span class="cm-string">"kiwifruit"</span>] <span class="cm-operator">&gt;</span>= <span class="cm-number">5</span>  ]</span>

				1

						<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">df</span>.<span class="cm-property">loc</span>[<span class="cm-variable">df</span>[<span class="cm-string">"kiwifruit"</span>] <span class="cm-operator">&gt;</span>= <span class="cm-number">5</span>  ]</span>

<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df</span>)&gt;&gt;&gt;</span>

				1

						<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df</span>)&gt;&gt;&gt;</span>

apple  orange  banana  strawberry  kiwifruit
1      6       8       6           3         10
5      8       2       5           4          8
8      6       8       4           8          8

a

apple orange banana strawberry kiwifruit

1 6 8 6 3 10

5 8 2 5 4 8

8 6 8 4 8 8

■内部結合　marge

<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">as</span> <span class="cm-variable">np</span></span>

				1

						<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">as</span> <span class="cm-variable">np</span></span>

<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span>

				1

						<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-variable">data1</span> = {<span class="cm-string">"fruits"</span>: [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"kiwifruit"</span>],</span>

				1

						<span role="presentation"><span class="cm-variable">data1</span> = {<span class="cm-string">"fruits"</span>: [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"kiwifruit"</span>],</span>

<span role="presentation">        <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2006</span>],</span>

				1

						<span role="presentation">        <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2006</span>],</span>

<span role="presentation">        <span class="cm-string">"amount"</span>: [<span class="cm-number">1</span>, <span class="cm-number">4</span>, <span class="cm-number">5</span>, <span class="cm-number">6</span>, <span class="cm-number">3</span>]}</span>

				1

						<span role="presentation">        <span class="cm-string">"amount"</span>: [<span class="cm-number">1</span>, <span class="cm-number">4</span>, <span class="cm-number">5</span>, <span class="cm-number">6</span>, <span class="cm-number">3</span>]}</span>

<span role="presentation"><span class="cm-variable">df1</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data1</span>)</span>

				1

						<span role="presentation"><span class="cm-variable">df1</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data1</span>)</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-variable">data2</span> = {<span class="cm-string">"fruits"</span>: [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"mango"</span>],</span>

				1

						<span role="presentation"><span class="cm-variable">data2</span> = {<span class="cm-string">"fruits"</span>: [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"mango"</span>],</span>

<span role="presentation">        <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2007</span>],</span>

				1

						<span role="presentation">        <span class="cm-string">"year"</span>: [<span class="cm-number">2001</span>, <span class="cm-number">2002</span>, <span class="cm-number">2001</span>, <span class="cm-number">2008</span>, <span class="cm-number">2007</span>],</span>

<span role="presentation">        <span class="cm-string">"price"</span>: [<span class="cm-number">150</span>, <span class="cm-number">120</span>, <span class="cm-number">100</span>, <span class="cm-number">250</span>, <span class="cm-number">3000</span>]}</span>

				1

						<span role="presentation">        <span class="cm-string">"price"</span>: [<span class="cm-number">150</span>, <span class="cm-number">120</span>, <span class="cm-number">100</span>, <span class="cm-number">250</span>, <span class="cm-number">3000</span>]}</span>

<span role="presentation"><span class="cm-variable">df2</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data2</span>)</span>

				1

						<span role="presentation"><span class="cm-variable">df2</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>(<span class="cm-variable">data2</span>)</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-comment"># df1, df2の中身を確認してください</span></span>

				1

						<span role="presentation"><span class="cm-comment"># df1, df2の中身を確認してください</span></span>

<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df1</span>)</span>

				1

						<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df1</span>)</span>

<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df2</span>)</span>

				1

						<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df2</span>)</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-comment"># df1とdf2を"fruits"をキーに内部結合して作成したDataFrameをdf3に代入してください</span></span>

				1

						<span role="presentation"><span class="cm-comment"># df1とdf2を"fruits"をキーに内部結合して作成したDataFrameをdf3に代入してください</span></span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-variable">df3</span>= <span class="cm-variable">pd</span>.<span class="cm-property">merge</span>(<span class="cm-variable">df1</span>,<span class="cm-variable">df2</span>,<span class="cm-variable">on</span>=<span class="cm-string">"fruits"</span>,<span class="cm-variable">how</span>=<span class="cm-string">"inner"</span>)</span>

				1

						<span role="presentation"><span class="cm-variable">df3</span>= <span class="cm-variable">pd</span>.<span class="cm-property">merge</span>(<span class="cm-variable">df1</span>,<span class="cm-variable">df2</span>,<span class="cm-variable">on</span>=<span class="cm-string">"fruits"</span>,<span class="cm-variable">how</span>=<span class="cm-string">"inner"</span>)</span>

<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df3</span>)</span>

				1

						<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df3</span>)</span>

   amount      fruits  year
0       1       apple  2001
1       4      orange  2002
2       5      banana  2001
3       6  strawberry  2008
4       3   kiwifruit  2006

amount fruits year

0 1 apple 2001

1 4 orange 2002

2 5 banana 2001

3 6 strawberry 2008

4 3 kiwifruit 2006

       fruits  price  year
0       apple    150  2001
1      orange    120  2002
2      banana    100  2001
3  strawberry    250  2008
4       mango   3000  2007

fruits price year

0 apple 150 2001

1 orange 120 2002

2 banana 100 2001

3 strawberry 250 2008

4 mango 3000 2007

   amount      fruits  year_x  price  year_y
0       1       apple    2001    150    2001
1       4      orange    2002    120    2002
2       5      banana    2001    100    2001
3       6  strawberry    2008    250    2008

amount fruits year_x price year_y

0 1 apple 2001 150 2001

1 4 orange 2002 120 2002

2 5 banana 2001 100 2001

3 6 strawberry 2008 250 2008

DataFrame連結縦連結は同じカラム横連結はaxis=1同じインデックス

2017年11月26日 by 河副太智 Leave a Comment

DataFrame同士を一定の方向についてそのままつなげる操作を連結
pandas.concat("DataFrameのリスト", axis=0)とすることでリストの先頭から順に縦方向に連結
axis=1を指定することで横方向に連結。

縦方向に連結するときは同じカラムについて連結され、
横方向に連結するときは同じインデックスについて連結されます。

<span role="presentation"><span class="cm-keyword">mport</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span>

				1

						<span role="presentation"><span class="cm-keyword">mport</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-comment"># 指定のインデックスとカラムを持つDataFrameを乱数によって作成する関数</span></span>

				1

						<span role="presentation"><span class="cm-comment"># 指定のインデックスとカラムを持つDataFrameを乱数によって作成する関数</span></span>

<span role="presentation"><span class="cm-keyword">def</span> <span class="cm-def">make_random_df</span>(<span class="cm-variable">index</span>, <span class="cm-variable">columns</span>, <span class="cm-variable">seed</span>):</span>

				1

						<span role="presentation"><span class="cm-keyword">def</span> <span class="cm-def">make_random_df</span>(<span class="cm-variable">index</span>, <span class="cm-variable">columns</span>, <span class="cm-variable">seed</span>):</span>

<span role="presentation">    <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">seed</span>(<span class="cm-variable">seed</span>)</span>

				1

						<span role="presentation">    <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">seed</span>(<span class="cm-variable">seed</span>)</span>

<span role="presentation">    <span class="cm-variable">df</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>()</span>

				1

						<span role="presentation">    <span class="cm-variable">df</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>()</span>

<span role="presentation">    <span class="cm-keyword">for</span> <span class="cm-variable">column</span> <span class="cm-keyword">in</span> <span class="cm-variable">columns</span>:</span>

				1

						<span role="presentation">    <span class="cm-keyword">for</span> <span class="cm-variable">column</span> <span class="cm-keyword">in</span> <span class="cm-variable">columns</span>:</span>

<span role="presentation">        <span class="cm-variable">df</span>[<span class="cm-variable">column</span>] = <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">choice</span>(<span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">101</span>), <span class="cm-builtin">len</span>(<span class="cm-variable">index</span>))</span>

				1

						<span role="presentation">        <span class="cm-variable">df</span>[<span class="cm-variable">column</span>] = <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">choice</span>(<span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">101</span>), <span class="cm-builtin">len</span>(<span class="cm-variable">index</span>))</span>

<span role="presentation">    <span class="cm-variable">df</span>.<span class="cm-property">index</span> = <span class="cm-variable">index</span></span>

				1

						<span role="presentation">    <span class="cm-variable">df</span>.<span class="cm-property">index</span> = <span class="cm-variable">index</span></span>

<span role="presentation">    <span class="cm-keyword">return</span> <span class="cm-variable">df</span></span>

				1

						<span role="presentation">    <span class="cm-keyword">return</span> <span class="cm-variable">df</span></span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-variable">columns</span> = [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>]</span>

				1

						<span role="presentation"><span class="cm-variable">columns</span> = [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>]</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-comment"># df_data1とdf_data2を縦方向に連結しdf1に代入</span></span>

				1

						<span role="presentation"><span class="cm-comment"># df_data1とdf_data2を縦方向に連結しdf1に代入</span></span>

<span role="presentation"><span class="cm-variable">df1</span>= <span class="cm-variable">pd</span>.<span class="cm-property">concat</span>([<span class="cm-variable">df_data1</span>,<span class="cm-variable">df_data2</span>])</span>

				1

						<span role="presentation"><span class="cm-variable">df1</span>= <span class="cm-variable">pd</span>.<span class="cm-property">concat</span>([<span class="cm-variable">df_data1</span>,<span class="cm-variable">df_data2</span>])</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-comment"># df_data1とdf_data2を横方向に連結しdf2に代入</span></span>

				1

						<span role="presentation"><span class="cm-comment"># df_data1とdf_data2を横方向に連結しdf2に代入</span></span>

<span role="presentation"><span class="cm-variable">df2</span> =<span class="cm-variable">pd</span>.<span class="cm-property">concat</span>([<span class="cm-variable">df_data1</span>,<span class="cm-variable">df_data2</span>], <span class="cm-variable">axis</span>=<span class="cm-number">1</span>)</span>

				1

						<span role="presentation"><span class="cm-variable">df2</span> =<span class="cm-variable">pd</span>.<span class="cm-property">concat</span>([<span class="cm-variable">df_data1</span>,<span class="cm-variable">df_data2</span>], <span class="cm-variable">axis</span>=<span class="cm-number">1</span>)</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df1</span>)</span>

				1

						<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df1</span>)</span>

<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df2</span>)</span>

				1

						<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df2</span>)</span>

   apple  orange  banana
1     45      10      71
2     48      84      89
3     65      22      89
4     68      37      13
5     68      88      59
1     38      76      17
2     13       6       2
3     73      80      77
4     10      65      72
   apple  orange  banana  apple  orange  banana
1     45      10      71   38.0    76.0    17.0
2     48      84      89   13.0     6.0     2.0
3     65      22      89   73.0    80.0    77.0
4     68      37      13   10.0    65.0    72.0
5     68      88      59    NaN     NaN     NaN

apple orange banana

1 45 10 71

2 48 84 89

3 65 22 89

4 68 37 13

5 68 88 59

1 38 76 17

2 13 6 2

3 73 80 77

4 10 65 72

apple orange banana apple orange banana

1 45 10 71 38.0 76.0 17.0

2 48 84 89 13.0 6.0 2.0

3 65 22 89 73.0 80.0 77.0

4 68 37 13 10.0 65.0 72.0

5 68 88 59 NaN NaN NaN

DataFrameの要素を条件に一致する行、列を取得

2017年11月25日 by 河副太智 Leave a Comment

<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">as</span> <span class="cm-variable">np</span></span>

				1

						<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">numpy</span> <span class="cm-keyword">as</span> <span class="cm-variable">np</span></span>

<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span>

				1

						<span role="presentation"><span class="cm-keyword">import</span> <span class="cm-variable">pandas</span> <span class="cm-keyword">as</span> <span class="cm-variable">pd</span></span>

<span role="presentation"><span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">seed</span>(<span class="cm-number">0</span>)</span>

				1

						<span role="presentation"><span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">seed</span>(<span class="cm-number">0</span>)</span>

<span role="presentation"><span class="cm-variable">columns</span> = [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"kiwifruit"</span>]</span>

				1

						<span role="presentation"><span class="cm-variable">columns</span> = [<span class="cm-string">"apple"</span>, <span class="cm-string">"orange"</span>, <span class="cm-string">"banana"</span>, <span class="cm-string">"strawberry"</span>, <span class="cm-string">"kiwifruit"</span>]</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-comment"># DataFrameを生成し、列を追加</span></span>

				1

						<span role="presentation"><span class="cm-comment"># DataFrameを生成し、列を追加</span></span>

<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>()</span>

				1

						<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">pd</span>.<span class="cm-property">DataFrame</span>()</span>

<span role="presentation"><span class="cm-keyword">for</span> <span class="cm-variable">column</span> <span class="cm-keyword">in</span> <span class="cm-variable">columns</span>:</span>

				1

						<span role="presentation"><span class="cm-keyword">for</span> <span class="cm-variable">column</span> <span class="cm-keyword">in</span> <span class="cm-variable">columns</span>:</span>

<span role="presentation">    <span class="cm-variable">df</span>[<span class="cm-variable">column</span>] = <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">choice</span>(<span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>), <span class="cm-number">10</span>)</span>

				1

						<span role="presentation">    <span class="cm-variable">df</span>[<span class="cm-variable">column</span>] = <span class="cm-variable">np</span>.<span class="cm-property">random</span>.<span class="cm-property">choice</span>(<span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>), <span class="cm-number">10</span>)</span>

<span role="presentation"><span class="cm-variable">df</span>.<span class="cm-property">index</span> = <span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>)</span>

				1

						<span role="presentation"><span class="cm-variable">df</span>.<span class="cm-property">index</span> = <span class="cm-builtin">range</span>(<span class="cm-number">1</span>, <span class="cm-number">11</span>)</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-comment"># dfの"apple"列が5以上かつ"kiwifruit"列が5以上の値をもつ行を含むDataFrameをdfに代入してください</span></span>

				1

						<span role="presentation"><span class="cm-comment"># dfの"apple"列が5以上かつ"kiwifruit"列が5以上の値をもつ行を含むDataFrameをdfに代入してください</span></span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">df</span>.<span class="cm-property">loc</span>[<span class="cm-variable">df</span>[<span class="cm-string">"apple"</span>] <span class="cm-operator">&gt;</span>= <span class="cm-number">5</span>]</span>

				1

						<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">df</span>.<span class="cm-property">loc</span>[<span class="cm-variable">df</span>[<span class="cm-string">"apple"</span>] <span class="cm-operator">&gt;</span>= <span class="cm-number">5</span>]</span>

<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">df</span>.<span class="cm-property">loc</span>[<span class="cm-variable">df</span><span class=" CodeMirror-matchingbracket">[</span><span class="cm-string">"kiwifruit"</span><span class=" CodeMirror-matchingbracket">]</span> <span class="cm-operator">&gt;</span>= <span class="cm-number">5</span>  ]</span>

				1

						<span role="presentation"><span class="cm-variable">df</span> = <span class="cm-variable">df</span>.<span class="cm-property">loc</span>[<span class="cm-variable">df</span><span class=" CodeMirror-matchingbracket">[</span><span class="cm-string">"kiwifruit"</span><span class=" CodeMirror-matchingbracket">]</span> <span class="cm-operator">&gt;</span>= <span class="cm-number">5</span>  ]</span>

<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df</span>)</span>

				1

						<span role="presentation"><span class="cm-builtin">print</span>(<span class="cm-variable">df</span>)</span>

   apple  orange  banana  strawberry  kiwifruit
1      6       8       6           3         10
5      8       2       5           4          8
8      6       8       4           8          8

apple orange banana strawberry kiwifruit

1 6 8 6 3 10

5 8 2 5 4 8

8 6 8 4 8 8

プログラミング

■データフレーム作成

​■データフレーム縦方向に直接結合

■データフレーム横方向に直接結合

■ソート

■フィルタリング 条件抽出

■内部結合 marge

`■データフレーム縦方向に直接結合`

`■データフレーム横方向に直接結合`

■フィルタリング　条件抽出

■内部結合　marge