Pandas

DataFrame dropで行や列を削除,　列削除時は第二引数にaxis=1

2017年12月15日 by 河副太智 Leave a Comment

データフレームの定義

>>> df = pd.DataFrame(np.arange(12).reshape(3,4),
                      columns=['A', 'B', 'C', 'D'])
>>> df
   A  B   C   D
0  0  1   2   3
1  4  5   6   7
2  8  9  10  11

>>> df = pd.DataFrame(np.arange(12).reshape(3,4),

columns=['A', 'B', 'C', 'D'])

>>> df

A B C D

0 0 1 2 3

1 4 5 6 7

2 8 9 10 11

Drop columnsを使う例

>>> df.drop(['B', 'C'], axis=1)
   A   D
0  0   3
1  4   7
2  8  11

>>> df.drop(['B', 'C'], axis=1)

A D

0 0 3

1 4 7

2 8 11

>>> df.drop(columns=['B', 'C'])
   A   D
0  0   3
1  4   7
2  8  11

>>> df.drop(columns=['B', 'C'])

A D

0 0 3

1 4 7

2 8 11

Drop a row by index

>>> df.drop([0, 1])
   A  B   C   D
2  8  9  10  11

>>> df.drop([0, 1])

A B C D

2 8 9 10 11

split()でデータフレームを任意の文字列で分割

2017年12月5日 by 河副太智 Leave a Comment

データフレームのカラムのcol1にa,bを交互に7個づつ振る

import pandas as pd
import datetime


df = pd.DataFrame({'dt1': [datetime.datetime(2014, 10, 1),
                           datetime.datetime(2014, 10, 2),
                           datetime.datetime(2014, 10, 3),
                           datetime.datetime(2014, 10, 4),
                           datetime.datetime(2014, 10, 5),
                           datetime.datetime(2014, 10, 6),
                           datetime.datetime(2014, 10, 7),
                           datetime.datetime(2014, 11, 1),
                           datetime.datetime(2014, 11, 2),
                           datetime.datetime(2014, 11, 3),
                           datetime.datetime(2014, 11, 4),
                           datetime.datetime(2014, 11, 5),
                           datetime.datetime(2014, 11, 6),
                           datetime.datetime(2014, 11, 7)],
                   'col1': 'a b'.split() * 7})
print(df)

x = df.groupby('col1').groups

print(x)

print (len((x["b"])))

import pandas as pd

import datetime

df = pd.DataFrame({'dt1': [datetime.datetime(2014, 10, 1),

datetime.datetime(2014, 10, 2),

datetime.datetime(2014, 10, 3),

datetime.datetime(2014, 10, 4),

datetime.datetime(2014, 10, 5),

datetime.datetime(2014, 10, 6),

datetime.datetime(2014, 10, 7),

datetime.datetime(2014, 11, 1),

datetime.datetime(2014, 11, 2),

datetime.datetime(2014, 11, 3),

datetime.datetime(2014, 11, 4),

datetime.datetime(2014, 11, 5),

datetime.datetime(2014, 11, 6),

datetime.datetime(2014, 11, 7)],

'col1': 'a b'.split() * 7})

print(df)

x = df.groupby('col1').groups

print(x)

print (len((x["b"])))

   col1        dt1
0     a 2014-10-01
1     b 2014-10-02
2     a 2014-10-03
3     b 2014-10-04
4     a 2014-10-05
5     b 2014-10-06
6     a 2014-10-07
7     b 2014-11-01
8     a 2014-11-02
9     b 2014-11-03
10    a 2014-11-04
11    b 2014-11-05
12    a 2014-11-06
13    b 2014-11-07
{'b': Int64Index([1, 3, 5, 7, 9, 11, 13], dtype='int64'), 'a': Int64Index([0, 2, 4, 6, 8, 10, 12], dtype='int64')}
7

col1 dt1

0 a 2014-10-01

1 b 2014-10-02

2 a 2014-10-03

3 b 2014-10-04

4 a 2014-10-05

5 b 2014-10-06

6 a 2014-10-07

7 b 2014-11-01

8 a 2014-11-02

9 b 2014-11-03

10 a 2014-11-04

11 b 2014-11-05

12 a 2014-11-06

13 b 2014-11-07

{'b': Int64Index([1, 3, 5, 7, 9, 11, 13], dtype='int64'), 'a': Int64Index([0, 2, 4, 6, 8, 10, 12], dtype='int64')}

x = df.groupby(‘col1’).groups

でcol1をベースとしたグループ分けされたリストを渡し、

print (len(x[“b”])
bにグループ分けされたものの個数だけ表示

groupby()でベースを決めてグループ分け

2017年12月5日 by 河副太智 Leave a Comment

データフレームのカラムのcol1にa,bを交互に振っていき
それをaとbのグループ分けにgroupby()を使用する。

import pandas as pd
import datetime


df = pd.DataFrame({'dt1': [datetime.datetime(2014, 10, 1),
                           datetime.datetime(2014, 10, 2),
                           datetime.datetime(2014, 10, 3),
                           datetime.datetime(2014, 10, 4),
                           datetime.datetime(2014, 10, 5),
                           datetime.datetime(2014, 10, 6),
                           datetime.datetime(2014, 10, 7),
                           datetime.datetime(2014, 11, 1),
                           datetime.datetime(2014, 11, 2),
                           datetime.datetime(2014, 11, 3),
                           datetime.datetime(2014, 11, 4),
                           datetime.datetime(2014, 11, 5),
                           datetime.datetime(2014, 11, 6),
                           datetime.datetime(2014, 11, 7)],
                   'col1': 'a b'.split() * 7})
print(df)

x = df.groupby('col1').groups

print(x)

print (len((x["b"])))

import pandas as pd

import datetime

df = pd.DataFrame({'dt1': [datetime.datetime(2014, 10, 1),

datetime.datetime(2014, 10, 2),

datetime.datetime(2014, 10, 3),

datetime.datetime(2014, 10, 4),

datetime.datetime(2014, 10, 5),

datetime.datetime(2014, 10, 6),

datetime.datetime(2014, 10, 7),

datetime.datetime(2014, 11, 1),

datetime.datetime(2014, 11, 2),

datetime.datetime(2014, 11, 3),

datetime.datetime(2014, 11, 4),

datetime.datetime(2014, 11, 5),

datetime.datetime(2014, 11, 6),

datetime.datetime(2014, 11, 7)],

'col1': 'a b'.split() * 7})

print(df)

x = df.groupby('col1').groups

print(x)

print (len((x["b"])))

   col1        dt1
0     a 2014-10-01
1     b 2014-10-02
2     a 2014-10-03
3     b 2014-10-04
4     a 2014-10-05
5     b 2014-10-06
6     a 2014-10-07
7     b 2014-11-01
8     a 2014-11-02
9     b 2014-11-03
10    a 2014-11-04
11    b 2014-11-05
12    a 2014-11-06
13    b 2014-11-07
{'b': Int64Index([1, 3, 5, 7, 9, 11, 13], dtype='int64'), 'a': Int64Index([0, 2, 4, 6, 8, 10, 12], dtype='int64')}
7

col1 dt1

0 a 2014-10-01

1 b 2014-10-02

2 a 2014-10-03

3 b 2014-10-04

4 a 2014-10-05

5 b 2014-10-06

6 a 2014-10-07

7 b 2014-11-01

8 a 2014-11-02

9 b 2014-11-03

10 a 2014-11-04

11 b 2014-11-05

12 a 2014-11-06

13 b 2014-11-07

{'b': Int64Index([1, 3, 5, 7, 9, 11, 13], dtype='int64'), 'a': Int64Index([0, 2, 4, 6, 8, 10, 12], dtype='int64')}

x = df.groupby(‘col1’).groups

でcol1をベースとしたグループ分けされたリストを渡し、

print (len(x[“b”])
bにグループ分けされたものの個数だけ表示

Seriesできること

2017年12月3日 by 河副太智 Leave a Comment

import pandas as pd

1	import pandas as pd

■通常の辞書

<span class="cm-variable">fruits</span> = {<span class="cm-string">"banana"</span>: <span class="cm-number">3</span>, <span class="cm-string">"orange"</span>: <span class="cm-number">2</span>}
<span class="cm-builtin">print</span>(<span class="cm-variable">pd</span>.<span class="cm-property">Series</span>(<span class="cm-variable">fruits</span>))

fruits = {"banana": 3, "orange": 2}

print(pd.Series(fruits))

`>>> banana 3 orange 2 dtype: int64`

■辞書どうしの結合

<span role="presentation">index = ["cup", "chair", "table", "kagaku", "pictre"]
data = [7012, 9406, 9609, 3824,9701]

</span>

index = ["cup", "chair", "table", "kagaku", "pictre"]

data = [7012, 9406, 9609, 3824,9701]

<span role="presentation"><span class="cm-variable">series</span> = <span class="cm-variable">pd</span>.<span class="cm-property">Series</span>(<span class="cm-variable">data</span>, <span class="cm-variable">index</span>=<span class="cm-variable">index</span>)</span>

1	<span role="presentation"><span class="cm-variable">series</span> = <span class="cm-variable">pd</span>.<span class="cm-property">Series</span>(<span class="cm-variable">data</span>, <span class="cm-variable">index</span>=<span class="cm-variable">index</span>)</span>

<span role="presentation"></span>print(<span class="cm-variable">series</span>)&gt;&gt;&gt;

1	<span role="presentation"></span>print(<span class="cm-variable">series</span>)>>>

cup       7012
chair     9406
table     9609
kagaku    3824
pictre    9701
dtype: int64

cup 7012

chair 9406

table 9609

kagaku 3824

pictre 9701

dtype: int64

■条件抽出　フィルタリング

<span role="presentation"><span class="cm-comment"># series内の要素のうち、値が5以上10未満の要素を含むSeriesを作り、seriesに再代入</span></span>

1	<span role="presentation"><span class="cm-comment"># series内の要素のうち、値が5以上10未満の要素を含むSeriesを作り、seriesに再代入</span></span>

<span role="presentation"><span class="cm-variable">series</span> = <span class="cm-variable">series</span>[<span class="cm-variable">series</span> <span class="cm-operator">&gt;</span>= <span class="cm-number">5</span>][<span class="cm-variable">series</span> <span class="cm-operator">&lt;</span> <span class="cm-number">10</span>]</span>

series = series[series >= 5][series < 10]

<span role="presentation"></span><span role="presentation"><span class="cm-builtin">print</span><span class=" CodeMirror-matchingbracket">(</span><span class="cm-variable">series</span><span class=" CodeMirror-matchingbracket">)</span></span>

1	<span role="presentation"></span><span role="presentation"><span class="cm-builtin">print</span><span class=" CodeMirror-matchingbracket">(</span><span class="cm-variable">series</span><span class=" CodeMirror-matchingbracket">)</span></span>

■インデックス参照,key,キーから取得

<span role="presentation"><span class="cm-comment"># seriesの2から4までの3つをitems1に代入

index = ["apple", "orange", "banana", "strawberry", "kiwifruit"]
data = [10, 5, 8, 12, 3]
series = pd.Series(data, index=index)</span></span>

# seriesの2から4までの3つをitems1に代入

index = ["apple", "orange", "banana", "strawberry", "kiwifruit"]

data = [10, 5, 8, 12, 3]

series = pd.Series(data, index=index)

items1 = series[1:4] 結果 orange 5 banana 8 strawberry 12 dtype: int64

■値,value,キーワードから取得

index = [“apple”, “orange”, “banana”, “strawberry”, “kiwifruit”]
data = [10, 5, 8, 12, 3]
series = pd.Series(data, index=index)

<span role="presentation"><span class="cm-variable">items2</span>=<span class="cm-variable">series</span>[[<span class="cm-string">"apple"</span>,<span class="cm-string">"banana"</span>,<span class="cm-string">"kiwifruit"</span>]]</span>

				1

						<span role="presentation"><span class="cm-variable">items2</span>=<span class="cm-variable">series</span>[[<span class="cm-string">"apple"</span>,<span class="cm-string">"banana"</span>,<span class="cm-string">"kiwifruit"</span>]]</span>

<span role="presentation">​</span>

				1

						<span role="presentation">​</span>

結果

				1

						結果

apple 10
banana 8
kiwifruit 3
dtype: int64

■ソート

index = [“a”, “b”, “c”, “d”, “e”]
data = [10, 5, 8, 12, 3]
series = pd.Series(data, index=index)

# seriesをインデックスに基づいてアルファベット順にソートしたSeriesをitems1に代入
items1 = series.sort_index()

# seriesをデータについて値の大きさを昇順にソートしたSeriesをitems2に代入
items2 = series.sort_values()

ブールインデックス

2017年12月3日 by 河副太智 Leave a Comment

import numpy as np

arr = np.array([2, 3, 4, 5, 6, 7])
print(arr % 2 == 0)>>>[ True False True False True False]

print(arr[arr%2==0])>>>[2 4 6]

print(arr[arr%2 == False])>>>[2 4 6]

bin ビンニング一定数値の範囲内にある物を探す

2017年11月29日 by 河副太智 Leave a Comment

<span class="c1"># 年齢と性別のデータ</span>
<span class="n">df</span> <span class="o">=</span> <span class="n">DataFrame</span><span class="p">([[</span><span class="mi">20</span><span class="p">,</span><span class="s2">"F"</span><span class="p">],[</span><span class="mi">22</span><span class="p">,</span><span class="s2">"M"</span><span class="p">],[</span><span class="mi">25</span><span class="p">,</span><span class="s2">"M"</span><span class="p">],[</span><span class="mi">27</span><span class="p">,</span><span class="s2">"M"</span><span class="p">],[</span><span class="mi">21</span><span class="p">,</span><span class="s2">"F"</span><span class="p">],[</span><span class="mi">23</span><span class="p">,</span><span class="s2">"M"</span><span class="p">],[</span><span class="mi">37</span><span class="p">,</span><span class="s2">"F"</span><span class="p">],[</span><span class="mi">31</span><span class="p">,</span><span class="s2">"M"</span><span class="p">],[</span><span class="mi">61</span><span class="p">,</span><span class="s2">"F"</span><span class="p">],[</span><span class="mi">45</span><span class="p">,</span><span class="s2">"M"</span><span class="p">],[</span><span class="mi">41</span><span class="p">,</span><span class="s2">"F"</span><span class="p">],[</span><span class="mi">32</span><span class="p">,</span><span class="s2">"M"</span><span class="p">]],</span> <span class="n">columns</span><span class="o">=</span><span class="p">[</span><span class="s2">"age"</span><span class="p">,</span> <span class="s2">"sex"</span><span class="p">])</span>
<span class="k">print</span> <span class="n">df</span>
<span class="sd">"""</span>
<span class="sd">   age sex</span>
<span class="sd">0   20   F</span>
<span class="sd">1   22   M</span>
<span class="sd">2   25   M</span>
<span class="sd">3   27   M</span>
<span class="sd">4   21   F</span>
<span class="sd">5   23   M</span>
<span class="sd">6   37   F</span>
<span class="sd">7   31   M</span>
<span class="sd">8   61   F</span>
<span class="sd">9   45   M</span>
<span class="sd">10  41   F</span>
<span class="sd">11  32   M</span>
<span class="sd">"""</span>

<span class="c1"># ビンに分割するときの値</span>
<span class="n">bins</span> <span class="o">=</span> <span class="p">[</span><span class="mi">18</span><span class="p">,</span> <span class="mi">25</span><span class="p">,</span> <span class="mi">35</span><span class="p">,</span> <span class="mi">60</span><span class="p">,</span> <span class="mi">100</span><span class="p">]</span>
<span class="c1"># ビンの名前</span>
<span class="n">group_names</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"Youth"</span><span class="p">,</span> <span class="s2">"YoungAdult"</span><span class="p">,</span> <span class="s2">"MiddleAged"</span><span class="p">,</span> <span class="s2">"Senior"</span><span class="p">]</span>
<span class="c1"># ビン化</span>
<span class="k">print</span> <span class="n">pd</span><span class="o">.</span><span class="n">cut</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">age</span><span class="p">,</span> <span class="n">bins</span><span class="p">,</span> <span class="n">labels</span><span class="o">=</span><span class="n">group_names</span><span class="p">)</span>
<span class="sd">"""</span>
<span class="sd">Categorical: </span>
<span class="sd">[Youth, Youth, Youth, YoungAdult, Youth, Youth, nan, YoungAdult, nan, nan, nan, YoungAdult]</span>
<span class="sd">Levels (4): Index(['Youth', 'YoungAdult', 'MiddleAged', 'Senior'], dtype=object)</span>
<span class="sd">"""</span>

<span class="c1"># dfにビンの列を追加</span>
<span class="n">df</span><span class="p">[</span><span class="s2">"bin"</span><span class="p">]</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">cut</span><span class="p">(</span><span class="n">df</span><span class="o">.</span><span class="n">age</span><span class="p">,</span> <span class="n">bins</span><span class="p">,</span> <span class="n">labels</span><span class="o">=</span><span class="n">group_names</span><span class="p">)</span>
<span class="k">print</span> <span class="n">df</span>
<span class="sd">"""</span>
<span class="sd">    age sex         bin</span>
<span class="sd">0    20   F       Youth</span>
<span class="sd">1    22   M       Youth</span>
<span class="sd">2    25   M       Youth</span>
<span class="sd">3    27   M  YoungAdult</span>
<span class="sd">4    21   F       Youth</span>
<span class="sd">5    23   M       Youth</span>
<span class="sd">6    37   F  MiddleAged</span>
<span class="sd">7    31   M  YoungAdult</span>
<span class="sd">8    61   F      Senior</span>
<span class="sd">9    45   M  MiddleAged</span>
<span class="sd">10   41   F  MiddleAged</span>
<span class="sd">11   32   M  YoungAdult</span>
<span class="sd">"""</span>

# 年齢と性別のデータ

df = DataFrame([[20,"F"],[22,"M"],[25,"M"],[27,"M"],[21,"F"],[23,"M"],[37,"F"],[31,"M"],[61,"F"],[45,"M"],[41,"F"],[32,"M"]], columns=["age", "sex"])

print df

"""

age sex

0 20 F

1 22 M

2 25 M

3 27 M

4 21 F

5 23 M

6 37 F

7 31 M

8 61 F

9 45 M

10 41 F

11 32 M

"""

# ビンに分割するときの値

bins = [18, 25, 35, 60, 100]

# ビンの名前

group_names = ["Youth", "YoungAdult", "MiddleAged", "Senior"]

# ビン化

print pd.cut(df.age, bins, labels=group_names)

"""

Categorical:

[Youth, Youth, Youth, YoungAdult, Youth, Youth, nan, YoungAdult, nan, nan, nan, YoungAdult]

Levels (4): Index(['Youth', 'YoungAdult', 'MiddleAged', 'Senior'], dtype=object)

"""

# dfにビンの列を追加

df["bin"] = pd.cut(df.age, bins, labels=group_names)

print df

"""

age sex bin

0 20 F Youth

1 22 M Youth

2 25 M Youth

3 27 M YoungAdult

4 21 F Youth

5 23 M Youth

6 37 F MiddleAged

7 31 M YoungAdult

8 61 F Senior

9 45 M MiddleAged

10 41 F MiddleAged

11 32 M YoungAdult

"""

Pandas

​ ■通常の辞書

>>> banana 3 orange 2 dtype: int64