kharukaのブログ~お金と技術とキャリア~

Edu Fin~金融×教育~若いうちからお金について学ぶってだいじ!学んだテクノロジーはみんなの財産。過去、現在、将来の人生についてのブログ

データ解析の基礎その1-HTMLからのデータの取り出し-

入門 Python 3

入門 Python 3

目次

前提条件

OS:Windows 10 64-bit, version 1607

Anaconda 4.4.0(Python 3.6 version 64-bit)インストール

NumPyのインストール

手順概要

1.コマンドプロンプトでIPython Notebookを起動します。

2.HTMLからのデータの取り出し

手順

1.コマンドプロンプトでIPython Notebookを起動します。

ipython notebook

2.HTMLからのデータの取り出し

2.1.コマンドプロンプトでhtml5libとbeautifulsoup4をインストールします。
pip install html5lib
pip install beautifulsoup4
2.2.pandasをpdとしてインポートします。
import pandas as pd
2.3.URLをurlに代入します。
url='https://www.fdic.gov/bank/individual/failed/banklist.html'

参考:

FDIC: Failed Bank List

2.4.htmlデーダをDataFrameリストに格納します。DataFrameリストの引数を指定してDataFrameを作成します。
dframe_list=pd.io.html.read_html(url)
dframe=dframe_list[0]
dframe

Out:

   Bank Name   City    ST  CERT    Acquiring Institution   Closing Date    Updated Date
0  Fayette County Bank Saint Elmo  IL  1802   United Fidelity Bank, fsb   26-May-17 26-Jul-17
1  Guaranty Bank, (d/b/a BestBank in Georgia & Mi... Milwaukee   WI  30003  First-Citizens Bank & Trust Company 5-May-17  26-Jul-17
2  First NBC Bank  New Orleans LA  58302  Whitney Bank    28-Apr-17 26-Jul-17
3  Proficio Bank   Cottonwood Heights  UT  35495  Cache Valley Bank   3-Mar-17  18-May-17
4  Seaway Bank and Trust Company Chicago IL  19328  State Bank of Texas 27-Jan-17 18-May-17
5  Harvest Community Bank  Pennsville  NJ  34951  First-Citizens Bank & Trust Company 13-Jan-17 18-May-17
6  Allied Bank Mulberry    AR  91 Today's Bank  23-Sep-16   17-Nov-16
7 The Woodbury Banking Company    Woodbury    GA  11297   United Bank 19-Aug-16   1-Jun-17
8 First CornerStone Bank  King of Prussia PA  35312   First-Citizens Bank & Trust Company 6-May-16    6-Sep-16
9 Trust Company Bank  Memphis TN  9956    The Bank of Fayette County  29-Apr-16   6-Sep-16
10    North Milwaukee State Bank  Milwaukee   WI  20364   First-Citizens Bank & Trust Company 11-Mar-16   13-Mar-17
11    Hometown National Bank  Longview    WA  35156   Twin City Bank  2-Oct-15    13-Apr-16
12    The Bank of Georgia Peachtree City  GA  35259   Fidelity Bank   2-Oct-15    24-Oct-16
13    Premier Bank    Denver  CO  34112   United Fidelity Bank, fsb   10-Jul-15   17-Aug-16
14    Edgebrook Bank  Chicago IL  57772   Republic Bank of Chicago    8-May-15    12-Jul-16
15    Doral Bank En Espanol   San Juan    PR  32102   Banco Popular de Puerto Rico    27-Feb-15   13-May-15
16    Capitol City Bank & Trust Company   Atlanta GA  33938   First-Citizens Bank & Trust Company 13-Feb-15   21-Apr-15
17    Highland Community Bank Chicago IL  20290   United Fidelity Bank, fsb   23-Jan-15   21-Apr-15
18    First National Bank of Crestview    Crestview   FL  17557   First NBC Bank  16-Jan-15   8-May-17
19    Northern Star Bank  Mankato MN  34983   BankVista   19-Dec-14   6-Jan-16
20    Frontier Bank, FSB D/B/A El Paseo Bank  Palm Desert CA  34738   Bank of Southern California, N.A.   7-Nov-14    10-Nov-16
21    The National Republic Bank of Chicago   Chicago IL  916 State Bank of Texas 24-Oct-14   6-Jan-16
22    NBRS Financial  Rising Sun  MD  4862    Howard Bank 17-Oct-14   26-Mar-15
23    GreenChoice Bank, fsb   Chicago IL  28462   Providence Bank, LLC    25-Jul-14   12-Dec-16
24    Eastside Commercial Bank    Conyers GA  58125   Community & Southern Bank   18-Jul-14   11-Jul-16
25    The Freedom State Bank  Freedom OK  12483   Alva State Bank & Trust Company 27-Jun-14   25-Mar-16
26    Valley Bank Fort Lauderdale FL  21793   Landmark Bank, National Association 20-Jun-14   29-Jun-15
27    Valley Bank Moline  IL  10450   Great Southern Bank 20-Jun-14   26-Jun-15
28    Slavie Federal Savings Bank Bel Air MD  32368   Bay Bank, FSB   30-May-14   12-Dec-16
29    Columbia Savings Bank   Cincinnati  OH  32284   United Fidelity Bank, fsb   23-May-14   10-Nov-16
...   ... ... ... ... ... ... ...
523   ANB Financial, NA   Bentonville AR  33901   Pulaski Bank and Trust Company  9-May-08    28-Aug-12
524   Hume Bank   Hume    MO  1971    Security Bank   7-Mar-08    28-Aug-12
525   Douglass National Bank  Kansas City MO  24660   Liberty Bank and Trust Company  25-Jan-08   26-Oct-12
526   Miami Valley Bank   Lakeview    OH  16848   The Citizens Banking Company    4-Oct-07    12-Sep-16
527   NetBank Alpharetta  GA  32575   ING DIRECT  28-Sep-07   28-Aug-12
528   Metropolitan Savings Bank   Pittsburgh  PA  35353   Allegheny Valley Bank of Pittsburgh 2-Feb-07    27-Oct-10
529   Bank of Ephraim Ephraim UT  1249    Far West Bank   25-Jun-04   9-Apr-08
530   Reliance Bank   White Plains    NY  26778   Union State Bank    19-Mar-04   9-Apr-08
531   Guaranty National Bank of Tallahassee   Tallahassee FL  26838   Hancock Bank of Florida 12-Mar-04   5-Jun-12
532   Dollar Savings Bank Newark  NJ  31330   No Acquirer 14-Feb-04   9-Apr-08
533   Pulaski Savings Bank    Philadelphia    PA  27203   Earthstar Bank  14-Nov-03   22-Jul-05
534   First National Bank of Blanchardville   Blanchardville  WI  11639   The Park Bank   9-May-03    5-Jun-12
535   Southern Pacific Bank   Torrance    CA  27094   Beal Bank   7-Feb-03    20-Oct-08
536   Farmers Bank of Cheneyville Cheneyville LA  16445   Sabine State Bank & Trust   17-Dec-02   20-Oct-04
537   Bank of Alamo   Alamo   TN  9961    No Acquirer 8-Nov-02    18-Mar-05
538   AmTrade International Bank En Espanol   Atlanta GA  33784   No Acquirer 30-Sep-02   11-Sep-06
539   Universal Federal Savings Bank  Chicago IL  29355   Chicago Community Bank  27-Jun-02   9-Apr-08
540   Connecticut Bank of Commerce    Stamford    CT  19183   Hudson United Bank  26-Jun-02   14-Feb-12
541   New Century Bank    Shelby Township MI  34979   No Acquirer 28-Mar-02   18-Mar-05
542   Net 1st National Bank   Boca Raton  FL  26652   Bank Leumi USA  1-Mar-02    9-Apr-08
543   NextBank, NA    Phoenix AZ  22314   No Acquirer 7-Feb-02    5-Feb-15
544   Oakwood Deposit Bank Co.    Oakwood OH  8966    The State Bank & Trust Company  1-Feb-02    25-Oct-12
545   Bank of Sierra Blanca   Sierra Blanca   TX  22002   The Security State Bank of Pecos    18-Jan-02   6-Nov-03
546   Hamilton Bank, NA En Espanol    Miami   FL  24382   Israel Discount Bank of New York    11-Jan-02   21-Sep-15
547   Sinclair National Bank  Gravette    AR  34248   Delta Trust & Bank  7-Sep-01    10-Feb-04
548   Superior Bank, FSB  Hinsdale    IL  32646   Superior Federal, FSB   27-Jul-01   19-Aug-14
549   Malta National Bank Malta   OH  6629    North Valley Bank   3-May-01    18-Nov-02
550   First Alliance Bank & Trust Co. Manchester  NH  34264   Southern New Hampshire Bank & Trust 2-Feb-01    18-Feb-03
551   National State Bank of Metropolis   Metropolis  IL  3815    Banterra Bank of Marion 14-Dec-00   17-Mar-05
552   Bank of Honolulu    Honolulu    HI  21029   Bank of the Orient  13-Oct-00   17-Mar-05
553 rows × 7 columns                         
2.5.DataFrameですので列を表示できます。
dframe.columns

Out:

Index(['Bank Name', 'City', 'ST', 'CERT', 'Acquiring Institution',
       'Closing Date', 'Updated Date'],
      dtype='object')

スクリーンショット

f:id:kharuka2016:20170818152341p:plainf:id:kharuka2016:20170818152346p:plain

Pythonではじめる機械学習 ―scikit-learnで学ぶ特徴量エンジニアリングと機械学習の基礎

Pythonではじめる機械学習 ―scikit-learnで学ぶ特徴量エンジニアリングと機械学習の基礎

参考:

udemy 実践Pythonデータサイエンス

www.udemy.com

Pythonスタートブック

Pythonスタートブック