My Own Cheatsheet

Install Python

1
2
3

brew install pypthon3
brew install jupyter
pip3 install spyder

参考引用：《Beginning Python - From Novice to Professional Third Edition 2017》
整理笔记：CK

简单抛出一个异常

1
2
3
4
5
6
7
8
9
10

# 指明异常类
>>> raise Exception
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
Exception
# 指明异常类，外加参数
>>> raise Exception('hyperdrive overload')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
Exception: hyperdrive overload

也可以自定义一个异常类

class SomeCustomException(Exception): pass

为什么需要异常处理：

1. 为什么不用if语句
   • 用try包住代码段可以一次捕获多个异常，用if语句要一个一个写分别处理。
2. 异常处理可以让程序即使发生致命错误也不必退出：
   • 在交互式的程序里，如果发生错误可以重试

     1 2 3 4 5 6

     try: x = int(input('Enter the first number: ')) y = int(input('Enter the second number: ')) print(x / y) except ZeroDivisionError: print("The second number can't be zero!")

     Here，尽管出错了但异常被处理了，没有继续向上抛出异常，程序也没有退出。
   • 尽管如此，如果异常发生在程序内部，还是抛出异常退出比较好，出错马上附带信息地退出，比留着隐患不知道什么时候异常退出要好。

异常传递方式

一层一层网上传，从异常出现的地方冒泡网上传，传到调用它的函数，直至传到main函数，如果都没有被处理则最终退出程序

传的过程中是包含情景信息的

处理方式：

如何处理异常写在except中：

继续抛出异常

1. raise 不带参数，在不知道如何处理异常的时候
2. raise 其他异常类，这样原来的异常就回保持在上下文中带入新的异常，成为新异常信息的一部分抛出去

   1 2 3 4 5 6 7 8 9 10 11 12 13 14

   try: ... 1/0 ... except ZeroDivisionError: ... raise ValueError ... Traceback (most recent call last): File "<stdin>", line 2, in <module> ZeroDivisionError: division by zero During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 4, in <module> ValueError

   捕获不同异常分别处理

   1 2 3 4 5 6 7 8 9 10 11 12

   >>> try: ... x = int(input('Enter the first number: ')) ... y = int(input('Enter the second number: ')) ... print(x / y) ... except ZeroDivisionError: ... print("The second number can't be zero!") ... except TypeError: ... print("That wasn't a number, was it?") ... Enter the first number: 1 Enter the second number: 0 The second number can't be zero!

• 慢慢感受到跟if statements 相比的优越性了，后面还有！

捕获不同异常同等处理

• 原理跟第2点差不多，只是多个异常类型写在一个tuple里，这样可以一并处理了。

  1 2 3 4 5 6

  try: x = int(input('Enter the first number: ')) y = int(input('Enter the second number: ')) print(x / y) except (ZeroDivisionError, TypeError, NameError): print('Your numbers were bogus ...')

将捕获的异常对象存到一个变量里

• 这样可以自己决定如何利用这个异常信息

1
2
3
4
5
6

try:
    x = int(input('Enter the first number: '))
    y = int(input('Enter the second number: '))
    print(x / y)
except (ZeroDivisionError, TypeError) as e:
    print(e)

捕获所有异常

• 虽然可以处理多个异常，但总有可能漏掉某些无法预估的异常，所以except后留空捕获所有异常

1
2
3
4
5
6

try:
    x = int(input('Enter the first number: '))
    y = int(input('Enter the second number: '))
    print(x / y)
except:
    print('Something wrong happened ...')

• ⚠️这样也有危险，因为这样会将一些本来没有预计到的异常情况，全部当作一直的异常一视同仁的处理了，例如 Ctrl-C 终止运行会被捕获并当作：’Something wrong happened …’ 处理。而用户退出并不算输入错误或者什么异常。

为了保证处理方法只针对主要的异常，最好用except Exception as e: 检查 e

1
2
3
4
5
6

try:
    x = int(input('Enter the first number: '))
    y = int(input('Enter the second number: '))
    print(x / y)
except Exception as e:
    print('e')

这样就不会去处理类似SystemExit 和 KeyboardInterrupt 的异常，因为他们是 BaseException 的子类，Exception 也是其子类。

设定没有异常执行什么

1
2
3
4
5
6
7
8
9
10

while True:
  try:
    x = int(input('Enter the first number: '))
    y = int(input('Enter the second number: '))
    value = x / y
    print('x / y is', value)
  except:
    print('Invalid input. Please try again.')
  else:
      break

• 根据5点，这里捕获一切异常，因此输入x或y的过程中即使Ctrl-C也是无法退出的，会当作非法输入处理⚠️，如果这不是你希望看到的，最好用except Exception as e:

打扫卫生

• 举了个没什么意义的例子：

  1 2 3 4 5 6 7 8

  try: x = 1 / 0 except NameError: print("Unknown variable") else: print("That went well!") finally: print('Cleaning up ...')

异常处理的哲学思想

• 求原谅比求允许容易
  比如下面的代码，对于大部分有值的情况，这段代码都要查询字典两次

  1 2 3 4 5

  def describe_person(person): print('Description of', person['name']) print('Age:', person['age']) if 'occupation' in person: print('Occupation:', person['occupation'])

  如果用异常处理，只要简单假设是有值的，只需要执行一遍，而且还有用else finally等辅助判断，非常高效

  1 2 3 4 5 6

  def describe_person(person): print('Description of', person['name']) print('Age:', person['age']) try: print('Occupation:', person['occupation']) except KeyError: pass

  因此Python中使用try/except 比 if/else 更自然。

警告warnings

警告不会退出程序

1
2
3

>>> from warnings import warn
>>> warn("I've got a bad feeling about this.")
__main__:1: UserWarning: I've got a bad feeling about this.

Django MTV

内容来源：Python and Django Full Stack Web Developer Bootcamp

翻译整理：CK

1. 模型及数据库

说明：

• We use Models to incorporate Database into a Django Project
• Django comes equipped with SQLite
• Django can connect to a variety of SQL engine backends
• In the setting.py file we can edit the ENGIN parameter used for DATABASE
• To create an actual model, we use a class structure inside the relevant applications models.py file
• Each attribute of the class represents a field, which is like a column name with constrains in SQL
• Each attribute has type of field and constraints.
• Models will reference each other to represent the table relationships.
• A foreign key denotes that the column coincides with a primary key of another table

步骤:

1. 在setting.py中注册 app: first_app

1
2
3
4
5
6
7
8
9

INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'first_app',
]

2. 添加model（数据表），atrribute（字段），字段类型，约束及外键

Edit first_app/models.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

from django.db import models


# Create your models here.


class Topic(models.Model):
top_name = models.CharField(max_length=246, unique=True)

def __str__(self):
return self.top_name


class Webpage(models.Model):
topic = models.ForeignKey(Topic, on_delete=models.CASCADE)
name = models.CharField(max_length=264, unique=True)
url = models.URLField(unique=True)

def __str__(self):
return self.name


class AccessRecord(models.Model):
name = models.ForeignKey(Webpage,on_delete=models.CASCADE)
date = models.DateField(unique=True)

def __str__(self):
return str(self.date)

3. 移植

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

root@38c80f8e29c3:/code# python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying auth.0008_alter_user_username_max_length... OK
Applying sessions.0001_initial... OK

1
2
3
4
5
6
7

root@38c80f8e29c3:/code# python manage.py makemigrations first_app
Migrations for 'first_app':
first_app/migrations/0001_initial.py
- Create model AccessRecord
- Create model Topic
- Create model Webpage
- Add field name to accessrecord

1
2
3
4
5

root@38c80f8e29c3:/code# python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, first_app, sessions
Running migrations:
Applying first_app.0001_initial... OK

4. 在对应app的admin.py中注册admin将要管理的模块：

1
2
3
4
5
6

from django.contrib import admin
from first_app.models import AccessRecord, Topic, Webpage
# Register your models here.
admin.site.register(AccessRecord)
admin.site.register(Topic)
admin.site.register(Webpage)

• 创建超级用户：（“root@38c80f8e29c3:/code” 说明当前主机是容器）

1
2
3
4
5
6

root@38c80f8e29c3:/code# python manage.py createsuperuser
Username (leave blank to use 'root'): ck
Email address: willxxx@gmail.com
Password:
Password (again):
Superuser created successfully.

5. 填充伪数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

import os
# Configure settings for project
# Need to run this before calling models from application!
os.environ.setdefault('DJANGO_SETTINGS_MODULE','first_project.settings')

import django
# Import settings
django.setup()

import random
from first_app.models import Topic,Webpage,AccessRecord
from faker import Faker

fakegen = Faker()
topics = ['Search','Social','Marketplace','News','Games']

def add_topic():
t = Topic.objects.get_or_create(top_name=random.choice(topics))[0]
t.save()
return t



def populate(N=5):
'''
Create N Entries of Dates Accessed
'''

for entry in range(N):

# Get Topic for Entry
top = add_topic()

# Create Fake Data for entry
fake_url = fakegen.url()
fake_date = fakegen.date()
fake_name = fakegen.company()

# Create new Webpage Entry
webpg = Webpage.objects.get_or_create(topic=top,url=fake_url,name=fake_name)[0]

# Create Fake Access Record for that page
# Could add more of these if you wanted...
accRec = AccessRecord.objects.get_or_create(name=webpg,date=fake_date)[0]


if __name__ == '__main__':
print("Populating the databases...Please Wait")
populate(20)
print('Populating Complete')

• 执行populate_first_app.py

1
2
3

root@c271100f853b:/code# python populate_first_app.py
Populating the databases...Please Wait
Populating Complete

6. 编辑Views

• 在 views 中 import 所有用到的 models
• 用 view 查询 model 获取所需数据
• 传送 model 结果到 template

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

from django.shortcuts import render
# from django.http import HttpResponse
from first_app.models import Topic, Webpage, AccessRecord

# Create your views here.


def index(request):
my_dict = {'insert_me': "Hello I am from first_app/index.html!"}
return render(request, 'first_app/index.html', context=my_dict)


def help(request):
my_dict = {'insert_me': "Hello I am from first_app/help.html!"}
return render(request, 'first_app/help.html', context=my_dict)


def access(request):
webpages_list = AccessRecord.objects.order_by('date')
date_dict = {'access_records': webpages_list}
return render(request, 'first_app/access.html', context=date_dict)

7. 编辑 template 来接收来自 model 的数据并按设计的式样展示。

用模版标记语言 template tagging 链接模型和Html page

Edit templates/first_app/access.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

<!DOCTYPE html>
{% load staticfiles %}
<html>
<head>
<meta charset="utf-8">
<title>Django Table</title>
<link rel="stylesheet" type="text/css" href="{% static "css/mystyle.css" %}">
<link rel="shortcut icon" href="{% static 'favicon/favicon.png' %}">
</head>
<body>
<h1>Hi welcome to Django MTV !</h1>
<h2>test<h2>
<div class="djangotwo">
{% if access_records %}
<table>
<thead>
<th>Site Name</th>
<th>Date Accessed</th>
</thead>
{% for acc in access_records %}
<tr>
<td>{{ acc.name }}</td>
<td>{{ acc.date }}</td>
</tr>
{% endfor %}
</table>
{% else %}
<p>NO ACCESS RECORDS FOUND!</p>
{% endif %}
</div>
</body>
</html>

8. 将相应的 URL 映射到新建的 View

Edit first_app/urls.py

1
2
3
4
5
6
7
8

from django.conf.urls import url
from first_app import views

urlpatterns = [
url(r'^$', views.index, name='index'),
url(r'^help/', views.help, name='help'),
url(r'^access/', views.access, name='access'),
]

9. 模板的重用

• 找出项目中重复出现的页面
• 创建 base.html 添加 base tag
• 在需要的地方扩展 base tag

Django Class Based View

Django Deployment

Demo on pythonanywhere

Option 1: Deploy to www.pythonanywhere.com

1. Collect the requirement of you development environment
   In development environment

   1 2	➜ git:(master) ✗ source activate dj (dj) ➜ git:(master) ✗ pip freeze > requirement.txt

2. Upload code to Github

3. Register an account on https://www.pythonanywhere.com

4. Create a Web App

5. Create Virtual Env:
   Open a Console

   1	mkvirtualenv --python=python3.6 dj

6. Download code and install required packages

   1 2 3 4

   git clone https://github.com/chunkai-meng/QualityHats-Django.git cd QualityHats-Django pip install -r requirements.txt python manage.py migrate

7. Create a supper user

   1 2	# if havn't created super user python manage.py createsuperuser <username>

8. Back to Deshboard Page - Web - Add a new web app

   • Go to “web app setup” Page, input the following setting (change to your own)
     • Virtualenv: /home/hustmck/.virtualenvs/myproj
     • Source code: /home/hustmck/QualityHats-Django
   • Edit WSGI configuration file:/var/www/hustmck_pythonanywhere_com_wsgi.py
     • delete the Hello World code
     • Edit the Django code block as below

       1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

       # +++++++++++ DJANGO +++++++++++ # To use your own django app use code like this: import os import sys # ## assuming your django settings file is at '/home/hustmck/mysite/mysite/settings.py' ## and your manage.py is is at '/home/hustmck/mysite/manage.py' path = '/home/hustmck/QualityHats-Django' if path not in sys.path: sys.path.append(path) os.chdir(path) os.environ.setdefault("DJANGO_SETTINGS_MODULE","qualityhats.settings") import django django.setup() #os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings' # ## then, for django >=1.5: from django.core.wsgi import get_wsgi_application application = get_wsgi_application() ## or, for older django <=1.4 #import django.core.handlers.wsgi #application = django.core.handlers.wsgi.WSGIHandler()

9. Reload your app

10. Open your URL (Open Dashboard - Web tab - the link is on the top)

Deploy to AWS

coming soon

Deploy to Heroku

Setting up Django Debug Toolbar

• install model

  1	pip install django-debug-toolbar

• Edit Setting.py

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

  INSTALLED_APPS = [ 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', # 1. ADD debug toolbar APP 'debug_toolbar', .... .... MIDDLEWARE = [ 'django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', # 2. Add Debug toolbar Middleware 'debug_toolbar.middleware.DebugToolbarMiddleware', ] .... .... # 3. Add internal IP INTERNAL_IPS = ['127.0.0.1']

• Edit <project_folder/url.py>

  1 2 3 4 5 6

  from django.conf import settings .... if settings.DEBUG is True: import debug_toolbar urlpatterns += [url(r'^__debug__/', include(debug_toolbar.urls))]

Source

A simple explain to TDD

Why should I do it:

• Tests have a short feedback loop, enabling you and your team to learn faster and adjust
• Less time is spent debugging, allowing you to spend more time writing code
• Tests act as documentation for your code!
• They improve code quality while reducing bugs
• After refactoring code, your tests will tell you whether the change has broken previously working code, and…
• Tests can prevent your hair line from receding!

This is how it works:

• Write a test. – The test will flesh out some functionality in your app
• Then, run the test – The test should fail, since there’s no code to make it pass.
• Write the code – To make the test pass
• Run the test – If it passes, you are confident that the code you’ve written meets the test requirements
• Refactor code – Remove duplication, prune large objects and make the code more readable. Re-run the tests every time you refactor the code
• Repeat – That’s it!

机器学习笔记
数据处理及分析的步骤过程和原理
【整理中】