original source : http://www.deekras.com/django-validations-during-form-processing.html

form validation에 대한 documentation이 좀 이해하기 힘든데 이 블로그는 간단명료학 잘 정리했다.

This post is mostly based on the Django Docs on Form and Field Validation. I reformatted the information in a way that feels easier to use.

There are 3 types of cleaning methods that are run during form processing. These are normally executed when you call the is_valid() method on a form.  (is_valid() runs validation routines for all fields on the form. When this method is called, if all fields contain valid data, it will:

  • return True
  • place the form’s data in its cleaned_data attribute.)

In general, any cleaning method can raise a ValidationError if there is a problem with the data it is processing; it passes the relevant information to the ValidationError constructor.

Steps of validation

The methods below are run in the order given, one field at a time. That is, for each field in the form (in the order they are declared in the form definition). Then the form.clean(), or its override, is executed regardless if the previous methods have raised errors. If the Field.clean() method raises a ValidationError, its field-specific cleaning methods are not called. However, the cleaning methods for all remaining fields are still executed.

Normally, the clean() method will be run and it will take care of the first three validations (to_python(), validate(), run_validators()). But you can customize any of them, and when the clean() method is executed, it will run the customized method.

1. to_python() method on a Field

  • WHAT IT DOES: It coerces the value to correct datatype and raises ValidationError if that is not possible. This method accepts the raw value from the widget and returns the converted value.
  • EXAMPLE: a FloatField will turn the data into a Python float or raise a ValidationError.
  • HANDLES ERRORS: raises ValidationError on any error
  • RETURNS: returns the converted value.

2. validate() method on a Field

  • WHAT IT DOES: handles field-specific validation that is not suitable for a validator. It takes a value that has been coerced to correct datatype and raises ValidationError on any error.
  • HANDLES ERRORS: raises ValidationError on any error
  • RETURNS: This method does not return anything and shouldn’t alter the value.
  • NOTES: You should override it to handle validation logic that you can’t or don’t want to put in a validator.

3. run_validators() method on a Field

  • WHAT IT DOES: runs all of the field’s validators
  • HANDLES ERRORS: aggregates all the errors into a single ValidationError.
  • RETURNS:
  • NOTES: You shouldn’t need to override this method.

4. The clean() method on a Field subclass.

  • WHAT IT DOES: This is responsible for running to_python, validate and run_validators in the correct order and propagating their errors.
  • HANDLES ERRORS: If, at any time, any of the methods raise ValidationError, the validation stops and that error is raised.
  • RETURNS: This method returns the clean data, which is then inserted into the cleaned_data dictionary of the form.

5. The clean_<fieldname>() method in a form subclass – where <fieldname> is replaced with the name of the form field attribute.

  • WHAT IT DOES: This method does any cleaning that is specific to that particular attribute, unrelated to the type of field that it is.
  • HOW TO USE: This method is not passed any parameters. You will need to look up the value of the field in self.cleaned_data and remember that it will be a Python object at this point, not the original string submitted in the form (it will be in cleaned_data because the general field clean() method, above, has already cleaned the data once).
  • HANDLES ERRORS:
  • RETURNS: the cleaned value obtained from cleaned_data – regardless of whether it changed anything or not.

6. The Form subclass’s clean() method.

NOTES: Also note that there are special considerations when overriding the clean() method of a ModelForm subclass. (see the ModelForm documentation for more information)

  • WHAT IT DOES: This method can perform any validation that requires access to multiple fields from the form at once.
  • EXAMPLE: Checks that if field A is supplied, field B must contain a valid email address and the like.
  • HOW TO USE: Since the field validation methods have been run by the time clean() is called, you also have access to the form’s errors attribute which contains all the errors raised by cleaning of individual fields.
  • HANDLES ERRORS: Note that any errors raised by your Form.clean() override will not be associated with any field in particular. They go into a special “field” (called __all__), which you can access via the non_field_errors() method if you need to. If you want to attach errors to a specific field in the form, you need to call add_error().
  • RETURNS: This method can return a completely different dictionary if it wishes, which will be used as the cleaned_data.

Raising ValidationError examples:

if not flag:
    raise ValidationError('Please submit flag') –  a simple example
    raise ValidationError(_('text: %(flag)s'), 
                            code='no flag', 
                            params={'flag': '42'},)

multiple errors can be created as a list

    raise ValidationError([
        ValidationError(_('Error 1'), code='error1'),
        ValidationError(_('Error 2'), code='error2'),
    ])

Writing Validators

There are many builtin validators that match the field type (ex: EmailValidator for EmailField). Those validators can be customized too. (ex: class EmailValidator([message=None, code=None, whitelist=None])

Here’s a sample custom validator:

from django.core.exceptions import ValidationError

def validate_even(value):
   if value % 2 != 0:
      raise ValidationError('%s is not an even number' % value)

Then, this validator can be used for any fields when setting up the models:

class MyModel(models.Model):
   even_field = models.IntegerField(validators=[validate_even])

It can also be used for forms:

class MyForm(forms.Form):
   even_field = forms.IntegerField(validators=[validate_even])

Validators will not be run automatically when you save a model, but if you are using a ModelForm, it will run your validators on any fields that are included in your form.

original source : http://www.deekras.com/django-validations-during-form-processing.html

There are 3 types of cleaning methods that are run during form processing. These are normally executed when you call the is_valid() method on a form.  (is_valid() runs validation routines for all fields on the form. When this method is called, if all fields contain valid data, it will:

  • return True
  • place the form’s data in its cleaned_data attribute.)

In general, any cleaning method can raise a ValidationError if there is a problem with the data it is processing; it passes the relevant information to the ValidationError constructor.

Steps of validation

The methods below are run in the order given, one field at a time. That is, for each field in the form (in the order they are declared in the form definition). Then the form.clean(), or its override, is executed regardless if the previous methods have raised errors. If the Field.clean() method raises a ValidationError, its field-specific cleaning methods are not called. However, the cleaning methods for all remaining fields are still executed.

Normally, the clean() method will be run and it will take care of the first three validations (to_python(), validate(), run_validators()). But you can customize any of them, and when the clean() method is executed, it will run the customized method.

1. to_python() method on a Field

  • WHAT IT DOES: It coerces the value to correct datatype and raises ValidationError if that is not possible. This method accepts the raw value from the widget and returns the converted value.
  • EXAMPLE: a FloatField will turn the data into a Python float or raise a ValidationError.
  • HANDLES ERRORS: raises ValidationError on any error
  • RETURNS: returns the converted value.

2. validate() method on a Field

  • WHAT IT DOES: handles field-specific validation that is not suitable for a validator. It takes a value that has been coerced to correct datatype and raises ValidationError on any error.
  • HANDLES ERRORS: raises ValidationError on any error
  • RETURNS: This method does not return anything and shouldn’t alter the value.
  • NOTES: You should override it to handle validation logic that you can’t or don’t want to put in a validator.

3. run_validators() method on a Field

  • WHAT IT DOES: runs all of the field’s validators
  • HANDLES ERRORS: aggregates all the errors into a single ValidationError.
  • RETURNS:
  • NOTES: You shouldn’t need to override this method.

4. The clean() method on a Field subclass.

  • WHAT IT DOES: This is responsible for running to_python, validate and run_validators in the correct order and propagating their errors.
  • HANDLES ERRORS: If, at any time, any of the methods raise ValidationError, the validation stops and that error is raised.
  • RETURNS: This method returns the clean data, which is then inserted into the cleaned_data dictionary of the form.

5. The clean_<fieldname>() method in a form subclass – where <fieldname> is replaced with the name of the form field attribute.

  • WHAT IT DOES: This method does any cleaning that is specific to that particular attribute, unrelated to the type of field that it is.
  • HOW TO USE: This method is not passed any parameters. You will need to look up the value of the field in self.cleaned_data and remember that it will be a Python object at this point, not the original string submitted in the form (it will be in cleaned_data because the general field clean() method, above, has already cleaned the data once).
  • HANDLES ERRORS:
  • RETURNS: the cleaned value obtained from cleaned_data – regardless of whether it changed anything or not.

6. The Form subclass’s clean() method.

  • NOTES: Also note that there are special considerations when overriding the clean() method of a ModelForm subclass. (see the ModelForm documentation for more information)
  • WHAT IT DOES: This method can perform any validation that requires access to multiple fields from the form at once.
  • EXAMPLE: Checks that if field A is supplied, field B must contain a valid email address and the like.
  • HOW TO USE: Since the field validation methods have been run by the time clean() is called, you also have access to the form’s errors attribute which contains all the errors raised by cleaning of individual fields.
  • HANDLES ERRORS: Note that any errors raised by your Form.clean() override will not be associated with any field in particular. They go into a special “field” (called __all__), which you can access via the non_field_errors() method if you need to. If you want to attach errors to a specific field in the form, you need to call add_error().
  • RETURNS: This method can return a completely different dictionary if it wishes, which will be used as the cleaned_data.

Raising ValidationError examples:

if not flag:
    raise ValidationError('Please submit flag') –  a simple example
    raise ValidationError(_('text: %(flag)s'), 
                            code='no flag', 
                            params={'flag': '42'},)

multiple errors can be created as a list

    raise ValidationError([
        ValidationError(_('Error 1'), code='error1'),
        ValidationError(_('Error 2'), code='error2'),
    ])

Writing Validators

There are many builtin validators that match the field type (ex: EmailValidator for EmailField). Those validators can be customized too. (ex: class EmailValidator([message=None, code=None, whitelist=None])

Here’s a sample custom validator:

from django.core.exceptions import ValidationError

def validate_even(value):
   if value % 2 != 0:
      raise ValidationError('%s is not an even number' % value)

Then, this validator can be used for any fields when setting up the models:

class MyModel(models.Model):
   even_field = models.IntegerField(validators=[validate_even])

It can also be used for forms:

class MyForm(forms.Form):
   even_field = forms.IntegerField(validators=[validate_even])

Validators will not be run automatically when you save a model, but if you are using a ModelForm, it will run your validators on any fields that are included in your form.

class Model(**kwargs)

The keyword arguments are the names of the fields you’ve defined on your model. Note that instantiating a model in no way touches your database; for that, you need to save().

Model의 __init__을 이용해 Model이 생성될 때 어떤작업을 하고 싶은 경우 __init__을 override하는 것보다 . 아래와 같이 class method를 이용해 처리할것을 추천한다. 

1. Add a classmethod on the model class:

from django.db import models

class Book(models.Model):
    title = models.CharField(max_length=100)

    @classmethod
    def create(cls, title):
        book = cls(title=title)
        # do something with the book
        return book

book = Book.create("Pride and Prejudice")

2. Add a method on a custom manager (usually preferred):

class BookManager(models.Manager):
    def create_book(self, title):
        book = self.create(title=title)
        # do something with the book
        return book

class Book(models.Model):
    title = models.CharField(max_length=100)

    objects = BookManager()

book = Book.objects.create_book("Pride and Prejudice")

Customizing model loading

Model이 loading될때 추가 작업을 하고싶은 경우 from_db()를 override한다.classmethod

Model.from_db(db, field_names, values)

참고자료) https://www.webforefront.com/django/modelmethodsandmetaoptions.html 하단부

Refreshing objects from database

If you delete a field from a model instance, accessing it again reloads the value from the database:

>>> obj = MyModel.objects.first()
>>> del obj.field
>>> obj.field  # Loads the field from the database

Model.refresh_from_db(using=None, fields=None)

if you need to reload a model’s values from the database, you can use the refresh_from_db() method.

Validating objects

There are three steps involved in validating a model:

  1. Validate the model fields – Model.clean_fields()
  2. Validate the model as a whole – Model.clean()
  3. Validate the field uniqueness – Model.validate_unique()

All three steps are performed when you call a model’s full_clean() method.


Model.full_clean(exclude=None, validate_unique=True)

This method calls Model.clean_fields(), Model.clean(), and Model.validate_unique() (if validate_unique is True) 유효성 검사중에 어디서든 ValidationError raise되면 유효성검사가 실패한것이 된다.

The optional exclude argument can be used to provide a list of field names that can be excluded from validation and cleaning. ModelForm uses this argument to exclude fields that aren’t present on your form from being validated since any errors raised could not be corrected by the user.

Note that full_clean() will not be called automatically when you call your model’s save() method. 

full_clean()의 호출방법의 예는 아래와 같다.

from django.core.exceptions import ValidationError
try:
    article.full_clean()
except ValidationError as e:
    # Do something based on the errors contained in e.message_dict.
    # Display them to a user, or handle them programmatically.
    pass


Model.clean_fields(exclude=None)

This method will validate all fields on your model. The optional exclude argument lets you provide a list of field names to exclude from validation. It will raise a ValidationError if any fields fail validation.

The second step full_clean() performs is to call Model.clean(). This method should be overridden to perform custom validation on your model.

Model.clean()

This method should be used to provide custom model validation, and to modify attributes on your model if desired. For instance, you could use it to automatically provide a value for a field, or to do validation that requires access to more than a single field: 아래와 같이 여러 항목을 이용해서 validate하는 경우 clean()에서 작업한다.

import datetime
from django.core.exceptions import ValidationError
from django.db import models
from django.utils.translation import gettext_lazy as _

class Article(models.Model):
    ...
    def clean(self):
        # Don't allow draft entries to have a pub_date.
        if self.status == 'draft' and self.pub_date is not None:
            raise ValidationError(_('Draft entries may not have a publication date.'))
        # Set the pub_date for published items if it hasn't been set already.
        if self.status == 'published' and self.pub_date is None:
            self.pub_date = datetime.date.today()

To assign exceptions to a specific field, instantiate the ValidationError with a dictionary, where the keys are the field names. 예를 들어 pub_date field의 문제의 경우 아래와 같이 해준다.

class Article(models.Model):
    ...
    def clean(self):
        # Don't allow draft entries to have a pub_date.
        if self.status == 'draft' and self.pub_date is not None:
            raise ValidationError({'pub_date': _('Draft entries may not have a publication date.')})

Model.clean()에서 여러 에러가 발생하는 경우 아래와 같이  you can also pass a dictionary mapping field names to errors:

raise ValidationError({
    'title': ValidationError(_('Missing title.'), code='required'),
    'pub_date': ValidationError(_('Invalid date.'), code='invalid'),
})

ModelForm에서 Meta.fields or Meta.exclude를 이용해 제외된 field의 ValidationError를 raise해도작동이 되지 않고 ValueError가 발생하게 되는데 이에 대한 우회방법은 아래와 같다. 

To work around this dilemma, instead override Model.clean_fields() as it receives the list of fields that are excluded from validation. For example:

class Article(models.Model):
    ...
    def clean_fields(self, exclude=None): #마치 제외된 field가 없는척한다.
        super().clean_fields(exclude=exclude)
        if self.status == 'draft' and self.pub_date is not None:
            if exclude and 'status' in exclude:
                raise ValidationError(
                    _('Draft entries may not have a publication date.')
                )
            else:
                raise ValidationError({
                    'status': _(
                        'Set status to draft if there is not a '
                        'publication date.'
                     ),
                })

Model.validate_unique(exclude=None)

model의 field중에 unique, unique_together ,  unique_for_date|month|year 이 설정되있는 경우 호출되며 어떻게 unique를 결정할지에 대한 내용이 들어 간다.



Saving objects

To save an object back to the database, call save():

Model.save(force_insert=False, force_update=False,using=DEFAULT_DB_ALIAS, update_fields=None)



Auto-incrementing primary keys

AutoField — an auto-incrementing primary key — then that auto-incremented value will be calculated and saved as an attribute on your object the first time you call save():

>>> b2 = Blog(name='Cheddar Talk', tagline='Thoughts on cheese.')
>>> b2.id     # Returns None, because b2 doesn't have an ID yet.
>>> b2.save()
>>> b2.id     # Returns the ID of your new object.

There’s no way to tell what the value of an ID will be before you call save()

For convenience, each model has an AutoField named id by default unless you explicitly specify primary_key=True on a field in your model. See the documentation for AutoField for more details. 특별히 특정 field에  primary_key=True 를 설정하지 않으면 id라는 AutoField 가 만들어 지게 된다.



Model.pk

자동으로 만들어진 AutoField이건 개발자가 primary_key=True 설정해 지정된 field이건 간에 간단히 pk 이름으로 접근할수 있다.



AutoField 로 정의 되어있더라도 개발자가 임의로 값을 할당할수 있다.

AutoField 이더라도 아래와 같이 처음 obj가 생성될때 넣어주면 이미 있지 않은 id라면 만들어진다. 이미 있던 id라면 내용이 update가 된다.

>>> b3 = Blog(id=3, name='Cheddar Talk', tagline='Thoughts on cheese.')
>>> b3.id     # Returns 3.
>>> b3.save()
>>> b3.id     # Returns 3.
b4 = Blog(id=3, name='Not Cheddar', tagline='Anything but cheese.')
b4.save()  # Overrides the previous blog with ID=3!

class Model(**kwargs)

The keyword arguments are the names of the fields you’ve defined on your model. Note that instantiating a model in no way touches your database; for that, you need to save().

Model의 __init__을 이용해 Model이 생성될 때 어떤작업을 하고 싶은 경우 __init__을 override하는 것보다 . 아래와 같이 class method를 이용해 처리할것을 추천한다. 

1. Add a classmethod on the model class:

from django.db import models

class Book(models.Model):
    title = models.CharField(max_length=100)

    @classmethod
    def create(cls, title):
        book = cls(title=title)
        # do something with the book
        return book

book = Book.create("Pride and Prejudice")

2. Add a method on a custom manager (usually preferred):

class BookManager(models.Manager):
    def create_book(self, title):
        book = self.create(title=title)
        # do something with the book
        return book

class Book(models.Model):
    title = models.CharField(max_length=100)

    objects = BookManager()

book = Book.objects.create_book("Pride and Prejudice")

Customizing model loading

Model이 loading될때 추가 작업을 하고싶은 경우 from_db()를 override한다.classmethod

Model.from_db(db, field_names, values)

참고자료) https://www.webforefront.com/django/modelmethodsandmetaoptions.html 하단부

Refreshing objects from database

If you delete a field from a model instance, accessing it again reloads the value from the database:

>>> obj = MyModel.objects.first()
>>> del obj.field
>>> obj.field  # Loads the field from the database

Model.refresh_from_db(using=None, fields=None)

if you need to reload a model’s values from the database, you can use the refresh_from_db() method.

Validating objects

There are three steps involved in validating a model:

  1. Validate the model fields – Model.clean_fields()
  2. Validate the model as a whole – Model.clean()
  3. Validate the field uniqueness – Model.validate_unique()

All three steps are performed when you call a model’s full_clean() method.


Model.full_clean(exclude=None, validate_unique=True)

This method calls Model.clean_fields(), Model.clean(), and Model.validate_unique() (if validate_unique is True) 유효성 검사중에 어디서든 ValidationError raise되면 유효성검사가 실패한것이 된다.

The optional exclude argument can be used to provide a list of field names that can be excluded from validation and cleaning. ModelForm uses this argument to exclude fields that aren’t present on your form from being validated since any errors raised could not be corrected by the user.

Note that full_clean() will not be called automatically when you call your model’s save() method. 

full_clean()의 호출방법의 예는 아래와 같다.

from django.core.exceptions import ValidationError
try:
    article.full_clean()
except ValidationError as e:
    # Do something based on the errors contained in e.message_dict.
    # Display them to a user, or handle them programmatically.
    pass


Model.clean_fields(exclude=None)

This method will validate all fields on your model. The optional exclude argument lets you provide a list of field names to exclude from validation. It will raise a ValidationError if any fields fail validation.

The second step full_clean() performs is to call Model.clean(). This method should be overridden to perform custom validation on your model.

Model.clean()

This method should be used to provide custom model validation, and to modify attributes on your model if desired. For instance, you could use it to automatically provide a value for a field, or to do validation that requires access to more than a single field: 아래와 같이 여러 항목을 이용해서 validate하는 경우 clean()에서 작업한다.

import datetime
from django.core.exceptions import ValidationError
from django.db import models
from django.utils.translation import gettext_lazy as _

class Article(models.Model):
    ...
    def clean(self):
        # Don't allow draft entries to have a pub_date.
        if self.status == 'draft' and self.pub_date is not None:
            raise ValidationError(_('Draft entries may not have a publication date.'))
        # Set the pub_date for published items if it hasn't been set already.
        if self.status == 'published' and self.pub_date is None:
            self.pub_date = datetime.date.today()

To assign exceptions to a specific field, instantiate the ValidationError with a dictionary, where the keys are the field names. 예를 들어 pub_date field의 문제의 경우 아래와 같이 해준다.

class Article(models.Model):
    ...
    def clean(self):
        # Don't allow draft entries to have a pub_date.
        if self.status == 'draft' and self.pub_date is not None:
            raise ValidationError({'pub_date': _('Draft entries may not have a publication date.')})

Model.clean()에서 여러 에러가 발생하는 경우 아래와 같이  you can also pass a dictionary mapping field names to errors:

raise ValidationError({
    'title': ValidationError(_('Missing title.'), code='required'),
    'pub_date': ValidationError(_('Invalid date.'), code='invalid'),
})

ModelForm에서 Meta.fields or Meta.exclude를 이용해 제외된 field의 ValidationError를 raise해도작동이 되지 않고 ValueError가 발생하게 되는데 이에 대한 우회방법은 아래와 같다. 

To work around this dilemma, instead override Model.clean_fields() as it receives the list of fields that are excluded from validation. For example:

class Article(models.Model):
    ...
    def clean_fields(self, exclude=None): #마치 제외된 field가 없는척한다.
        super().clean_fields(exclude=exclude)
        if self.status == 'draft' and self.pub_date is not None:
            if exclude and 'status' in exclude:
                raise ValidationError(
                    _('Draft entries may not have a publication date.')
                )
            else:
                raise ValidationError({
                    'status': _(
                        'Set status to draft if there is not a '
                        'publication date.'
                     ),
                })

Model.validate_unique(exclude=None)

model의 field중에 unique, unique_together ,  unique_for_date|month|year 이 설정되있는 경우 호출되며 어떻게 unique를 결정할지에 대한 내용이 들어 간다.



Saving objects

To save an object back to the database, call save():

Model.save(force_insert=False, force_update=False,using=DEFAULT_DB_ALIAS, update_fields=None)



Auto-incrementing primary keys

AutoField — an auto-incrementing primary key — then that auto-incremented value will be calculated and saved as an attribute on your object the first time you call save():

>>> b2 = Blog(name='Cheddar Talk', tagline='Thoughts on cheese.')
>>> b2.id     # Returns None, because b2 doesn't have an ID yet.
>>> b2.save()
>>> b2.id     # Returns the ID of your new object.

There’s no way to tell what the value of an ID will be before you call save()

For convenience, each model has an AutoField named id by default unless you explicitly specify primary_key=True on a field in your model. See the documentation for AutoField for more details. 특별히 특정 field에  primary_key=True 를 설정하지 않으면 id라는 AutoField 가 만들어 지게 된다.



Model.pk

자동으로 만들어진 AutoField이건 개발자가 primary_key=True 설정해 지정된 field이건 간에 간단히 pk 이름으로 접근할수 있다.



AutoField 로 정의 되어있더라도 개발자가 임의로 값을 할당할수 있다.

AutoField 이더라도 아래와 같이 처음 obj가 생성될때 넣어주면 이미 있지 않은 id라면 만들어진다. 이미 있던 id라면 내용이 update가 된다.

>>> b3 = Blog(id=3, name='Cheddar Talk', tagline='Thoughts on cheese.')
>>> b3.id     # Returns 3.
>>> b3.save()
>>> b3.id     # Returns 3.
b4 = Blog(id=3, name='Not Cheddar', tagline='Anything but cheese.')
b4.save()  # Overrides the previous blog with ID=3!

form validation doc https://docs.djangoproject.com/en/3.0/ref/forms/validation/

참고 블로그) 내용이 official doc보다 간결하다. http://www.deekras.com/django-validations-during-form-processing.html

These are normally executed when you call the is_valid() method on a form. There are other things that can also trigger cleaning and validation (accessing the errors attribute or calling full_clean() directly), but normally they won’t be needed. 

(form validation은 form의 is_valid()를 통해 수행된다. form에 있는 errors에 접근하거나 full_clean()을 통해 수행되기도 한다. full_clean()는 model validation에서 사용되는 method이름과 같다.)

  •  to_python() 스트링을 python data type으로 전환한다.
  • validate() method on a Field handles field-specific validation that is not suitable for a validator. 
  • run_validators() method on a Field runs all of the field’s validators and aggregates all the errors into a single ValidationError. You shouldn’t need to override this method.
  • clean() method on a Field subclass is responsible for running to_python(), validate(), and run_validators() in the correct order and propagating their errors. This method returns the clean data, which is then inserted into the cleaned_data dictionary of the form.
  • clean_<fieldname>() method is called on a form subclass. This method does any cleaning that is specific to that particular attribute, unrelated to the type of field that it is. This method is not passed any parameters. You will need to look up the value of the field in self.cleaned_data and remember that it will be a Python object at this point, not the original string submitted in the form (it will be in cleaned_data because the general field clean() method, above, has already cleaned the data once).

기본적으로 위와 같은 순서로 진행되며 각 단계에서 무제가 발생하면 ValidationError를 raise한다.

Raising ValidationError

# Good
ValidationError(_('Invalid value'), code='invalid')

# Bad
ValidationError(_('Invalid value'))

# Good
ValidationError(
    _('Invalid value: %(value)s'),
    params={'value': '42'},
)

# Bad
ValidationError(_('Invalid value: %s') % value)

# Good
ValidationError(
    _('Invalid value: %(value)s'),
    params={'value': '42'},
)

# Bad
ValidationError(
    _('Invalid value: %s'),
    params=('42',),
)

Wrap the message with gettext to enable translation: _() 를 말한다.

# Good
ValidationError(_('Invalid value'))

# Bad
ValidationError('Invalid value')

Putting it all together:

raise ValidationError(
    _('Invalid value: %(value)s'),
    code='invalid',
    params={'value': '42'},)

Raising multiple errors

# Good
raise ValidationError([
    ValidationError(_('Error 1'), code='error1'),
    ValidationError(_('Error 2'), code='error2'),
])

# Bad
raise ValidationError([
    _('Error 1'),
    _('Error 2'),
])

기존 field를 extends해서 새로운 field를 만들고 새로운 validation rule을 지정하는 방법

1방법)

from django.core import validators
from django.forms import CharField

class SlugField(CharField):
    default_validators = [validators.validate_slug]
slug = forms.SlugField()

2방법)

slug = forms.CharField(validators=[validators.validate_slug])

validator만드는 방법 https://docs.djangoproject.com/en/3.0/ref/validators/

form의 field에 validation을 customize하는 방법 (to_python , validate)

from django import forms
from django.core.validators import validate_email

class MultiEmailField(forms.Field):
    def to_python(self, value):
        """Normalize data to a list of strings."""
        # Return an empty list if no input was given.
        if not value:
            return []
        return value.split(',')

    def validate(self, value):
        """Check if value consists only of valid emails."""
        # Use the parent's handling of required fields, etc.
        super().validate(value)
        for email in value:
            validate_email(email)
class ContactForm(forms.Form):
    subject = forms.CharField(max_length=100)
    message = forms.CharField()
    sender = forms.EmailField()
    recipients = MultiEmailField()
    cc_myself = forms.BooleanField(required=False)

Use MultiEmailField like any other form field. When the is_valid() method is called on the form, the MultiEmailField.clean() method will be run as part of the cleaning process and it will, in turn, call the custom to_python() and validate() methods.

form 단위에서 field validation 을 customize 하는 경우

from django import forms

class ContactForm(forms.Form):
    # Everything as before.
    ...

    def clean_recipients(self):
        data = self.cleaned_data['recipients']
        if "fred@example.com" not in data:
            raise forms.ValidationError("You have forgotten about Fred!")

        # Always return a value to use as the new cleaned data, even if
        # this method didn't change it.
        return data

이미 fileld단위의 validation이 끝났기 때문에 cleaned_data에 접근 가능하다.

Cleaning and validating fields that depend on each other (form 단위에서 여러개의 fields을 이용해서 validation하는 경우)

from django import forms

class ContactForm(forms.Form):
    # Everything as before.
    ...

    def clean(self):
        cleaned_data = super().clean()
        cc_myself = cleaned_data.get("cc_myself")
        subject = cleaned_data.get("subject")

        if cc_myself and subject:
            # Only do something if both fields are valid so far.
            if "help" not in subject:
                raise forms.ValidationError(
                    "Did not send for 'help' in the subject despite "
                    "CC'ing yourself."
                )

In this code, if the validation error is raised, the form will display an error message at the top of the form (normally) describing the problem.

The call to super().clean() in the example code ensures that any validation logic in parent classes is maintained. If your form inherits another that doesn’t return a cleaned_data dictionary in its clean() method (doing so is optional), then don’t assign cleaned_data to the result of the super() call and use self.cleaned_data instead:

def clean(self):
    super().clean()
    cc_myself = self.cleaned_data.get("cc_myself")
    ...
from django import forms
class ContactForm(forms.Form):
    # Everything as before.
    ...

    def clean(self):
        cleaned_data = super().clean()
        cc_myself = cleaned_data.get("cc_myself")
        subject = cleaned_data.get("subject")

        if cc_myself and subject and "help" not in subject:
            msg = "Must put 'help' in subject when cc'ing yourself."
            self.add_error('cc_myself', msg)
            self.add_error('subject', msg)

https://www.coursera.org/learn/machine-learning/lecture/OVM4M/deciding-what-to-try-next

Deciding What to Try Next

image

machine learning의 성능개선을 위해 해볼수 있는 조치들이다.

image


https://www.coursera.org/learn/machine-learning/lecture/yfbJY/evaluating-a-hypothesis

Evaluating a Hypothesis

image

overfitting을 방지하는 방법중의 하나로 데이터를 둘로 구분해서 한 부분( 70% )에 대한 Hypothesis (쎄타)값을 얻고 나머지 데이터에 적용해 보는 것이 있다.

image
image

https://www.coursera.org/learn/machine-learning/supplement/aFpD3/evaluating-a-hypothesis

Evaluating a Hypothesis

image
image


https://www.coursera.org/learn/machine-learning/lecture/QGKbr/model-selection-and-train-validation-test-sets

Model Selection and Train/Validation/Test Sets

image

 여러가지의  model을 생각할수 있다. 모든 features를 넣은 모델도 생각할수 있지만 적은 features만으로도 더 좋은 성능을 보이는 model이 있을수 있으며 이를 사용한다. 각각 model의 J() 최소값을 비교해서 결정할수 있다. 다만 모든 data를 training set으로 사용하면 그 data에 최적화된 model을 구할수 있지만 다른 새로운 data에 잘 부합하는지 확인할 방법이 없다. 확인을 위해 또 다른 data가 추가로 필요하다. 그러므로 주어진 data를 training set, cross validation set, test set으로 구분해서 각각이용하는 것이 좋다.  

image
image

위의 그림에서는 cross validation J()값이 제일 작은 d4를 model을 사용하는 것을 보여준다.

https://www.coursera.org/learn/machine-learning/supplement/XHQqO/model-selection-and-train-validation-test-sets

Model Selection and Train/Validation/Test Sets

image


https://www.coursera.org/learn/machine-learning/lecture/yCAup/diagnosing-bias-vs-variance

Diagnosing Bias vs. Variance

image
image

cross validation J() 를 대신해서 test J() 사용해도 된다.   

image

https://www.coursera.org/learn/machine-learning/supplement/81vp0/diagnosing-bias-vs-variance

Diagnosing Bias vs. Variance

image
image


https://www.coursera.org/learn/machine-learning/lecture/4VDlf/regularization-and-bias-variance

Regularization and Bias/Variance

image
image

training J() – (이 함수에 lamda가 있는데 유의)를 통해 최적의 theta를 구한다 그리고 또 이를 cross validation J() –(이함수에는 lamda가 없음에 유의)에 대입해서 가장 낮은 손실값을 가지는 theta를 구한다. 또 이를 test J() –(이함수에는 lamda가 없음에 유의) 에 대입해 본다.

image

https://www.coursera.org/learn/machine-learning/supplement/JPJJj/regularization-and-bias-variance

Regularization and Bias/Variance

image
image


https://www.coursera.org/learn/machine-learning/lecture/Kont7/learning-curves

Learning Curves

image
image

cross validation J()와 training J()가 높은 error에서 만난다는 것에 유의한다. 위 그림 좌측 하단 참조

image

https://www.coursera.org/learn/machine-learning/supplement/79woL/learning-curves

Learning Curves

image
image


https://www.coursera.org/learn/machine-learning/lecture/zJTzp/deciding-what-to-do-next-revisited

Deciding What to Do Next Revisited

training J() , cross validation J() 의 결과를 보고 어떤 조치를 해야 하는지에 대한 방법 설명

image

각 상황에 어떤 조치를 취해야 하는지 위에 그림 하단을 참조한다.

image


https://www.coursera.org/learn/machine-learning/lecture/4h5X4/prioritizing-what-to-work-on

Error Analysis

image

andrew 교수는 지저분하고 거칠수도 있지만 간단히 사용할수 있는 알고리즘을 통해 프로젝트에 접근하는 것을 추천한다.

image

이 강의에서는 spam 방지 알고리즘을 예로 들고 있다. 에러 분석 과정에서는 cross validation set에서 error를 일으킨(예상값과 실제값이 다른경우) 데이타를 category별로 구분하고 그 갯수를 확인한다. 그리고 이를 통해 어떤 새로운 방법을 적용할지 생각해 낸다.

image

알고리즘을 보완해 줄것이라고 예상되는 방법을 실제 적용해 보고 결과 수치가 어떻게 변화하는지 확인한다.

https://www.coursera.org/learn/machine-learning/supplement/Z11RP/error-analysis

Error Analysis

image
image


https://www.coursera.org/learn/machine-learning/lecture/tKMWX/error-metrics-for-skewed-classes

Error Metrics for Skewed Classes

본래 발생 가능성이 낮은 경우를 예측하는 경우 몇번의 우연한 예측이 들어 맞아도 좋은 성능을 보이는 알고리즘으로 평가 될수 있는데 이는 정확하지 않는 평가이다. 이와 같은 경우를 skewed class라고 하며 이를 평가할때는 precision과 recall 수치를 이용한다.

image


https://www.coursera.org/learn/machine-learning/lecture/CuONQ/trading-off-precision-and-recall

Trading Off Precision and Recall

image

cancer 여부를 판단하는 알고리즘을 예로 들고 있다. 예를 들어 계산값 판단기준을 0.99 와 같이 높게 잡으면 알고리즘의 정확성은 올라가지만  ( precision 값이 높아짐 ) 많은 경우 이미 암에 걸린 사람도 아닌것으로 판단될 가능성이 있다( recall 값이 낮아짐 ). 반대로 기준값을 0.01과 같이 낮추면 대부분의 사람이 암으로 판단되므로 정확성은 낮아 지고 recall값은 높아지게 된다. 그러므로 적당한 기준값을 설정해야 하며 이는 precision과 recall 값 둘다 적당히 유지 되어야 한다. 두 값을 적절히 조합한 수치가 f1 score이며 이를 알고리즘 성능 판단기준으로 삼는다.

image

기준값을 결정하는 방법으로 F1 score를 이용할수 있다. 


https://www.coursera.org/learn/machine-learning/lecture/XcNcz/data-for-machine-learning

Data For Machine Learning

대부분의 경우에는 많은 양의 training set이 머신러닝의 성능을 높이는데 도움이 된다.그렇지만 꼭 많은 데이터가 있어야만 하는 것은 아니다. 

image

대개 사람이 바로 대답할수 있는 질문의 경우 많은 양의 데이터가 필요하지 않다. 위의 그림의 two 단어 채우기 같은 작업을 들수 있다. 

많은 features를 가지고 있는 linear regression, logistic regression 많은 hidden layer를 가지고 있는 neural network의 경우 많은 데이터가 필요하다. 

https://www.coursera.org/learn/machine-learning/lecture/OVM4M/deciding-what-to-try-next

Deciding What to Try Next

image

machine learning의 성능개선을 위해 해볼수 있는 조치들이다.

image


https://www.coursera.org/learn/machine-learning/lecture/yfbJY/evaluating-a-hypothesis

Evaluating a Hypothesis

image

overfitting을 방지하는 방법중의 하나로 데이터를 둘로 구분해서 한 부분( 70% )에 대한 Hypothesis (쎄타)값을 얻고 나머지 데이터에 적용해 보는 것이 있다.

image
image

https://www.coursera.org/learn/machine-learning/supplement/aFpD3/evaluating-a-hypothesis

Evaluating a Hypothesis

image
image


https://www.coursera.org/learn/machine-learning/lecture/QGKbr/model-selection-and-train-validation-test-sets

Model Selection and Train/Validation/Test Sets

image

 여러가지의  model을 생각할수 있다. 모든 features를 넣은 모델도 생각할수 있지만 적은 features만으로도 더 좋은 성능을 보이는 model이 있을수 있으며 이를 사용한다. 각각 model의 J() 최소값을 비교해서 결정할수 있다. 다만 모든 data를 training set으로 사용하면 그 data에 최적화된 model을 구할수 있지만 다른 새로운 data에 잘 부합하는지 확인할 방법이 없다. 확인을 위해 또 다른 data가 추가로 필요하다. 그러므로 주어진 data를 training set, cross validation set, test set으로 구분해서 각각이용하는 것이 좋다.  

image
image

위의 그림에서는 cross validation J()값이 제일 작은 d4를 model을 사용하는 것을 보여준다.

https://www.coursera.org/learn/machine-learning/supplement/XHQqO/model-selection-and-train-validation-test-sets

Model Selection and Train/Validation/Test Sets

image


https://www.coursera.org/learn/machine-learning/lecture/yCAup/diagnosing-bias-vs-variance

Diagnosing Bias vs. Variance

image
image

cross validation J() 를 대신해서 test J() 사용해도 된다.   

image

https://www.coursera.org/learn/machine-learning/supplement/81vp0/diagnosing-bias-vs-variance

Diagnosing Bias vs. Variance

image
image


https://www.coursera.org/learn/machine-learning/lecture/4VDlf/regularization-and-bias-variance

Regularization and Bias/Variance

image
image

training J() – (이 함수에 lamda가 있는데 유의)를 통해 최적의 theta를 구한다 그리고 또 이를 cross validation J() –(이함수에는 lamda가 없음에 유의)에 대입해서 가장 낮은 손실값을 가지는 theta를 구한다. 또 이를 test J() –(이함수에는 lamda가 없음에 유의) 에 대입해 본다.

image

https://www.coursera.org/learn/machine-learning/supplement/JPJJj/regularization-and-bias-variance

Regularization and Bias/Variance

image
image


https://www.coursera.org/learn/machine-learning/lecture/Kont7/learning-curves

Learning Curves

image
image

cross validation J()와 training J()가 높은 error에서 만난다는 것에 유의한다. 위 그림 좌측 하단 참조

image

https://www.coursera.org/learn/machine-learning/supplement/79woL/learning-curves

Learning Curves

image
image


https://www.coursera.org/learn/machine-learning/lecture/zJTzp/deciding-what-to-do-next-revisited

Deciding What to Do Next Revisited

training J() , cross validation J() 의 결과를 보고 어떤 조치를 해야 하는지에 대한 방법 설명

image

각 상황에 어떤 조치를 취해야 하는지 위에 그림 하단을 참조한다.

image


https://www.coursera.org/learn/machine-learning/lecture/4h5X4/prioritizing-what-to-work-on

Error Analysis

image

andrew 교수는 지저분하고 거칠수도 있지만 간단히 사용할수 있는 알고리즘을 통해 프로젝트에 접근하는 것을 추천한다.

image

이 강의에서는 spam 방지 알고리즘을 예로 들고 있다. 에러 분석 과정에서는 cross validation set에서 error를 일으킨(예상값과 실제값이 다른경우) 데이타를 category별로 구분하고 그 갯수를 확인한다. 그리고 이를 통해 어떤 새로운 방법을 적용할지 생각해 낸다.

image

알고리즘을 보완해 줄것이라고 예상되는 방법을 실제 적용해 보고 결과 수치가 어떻게 변화하는지 확인한다.

https://www.coursera.org/learn/machine-learning/supplement/Z11RP/error-analysis

Error Analysis

image
image


https://www.coursera.org/learn/machine-learning/lecture/tKMWX/error-metrics-for-skewed-classes

Error Metrics for Skewed Classes

본래 발생 가능성이 낮은 경우를 예측하는 경우 몇번의 우연한 예측이 들어 맞아도 좋은 성능을 보이는 알고리즘으로 평가 될수 있는데 이는 정확하지 않는 평가이다. 이와 같은 경우를 skewed class라고 하며 이를 평가할때는 precision과 recall 수치를 이용한다.

image


https://www.coursera.org/learn/machine-learning/lecture/CuONQ/trading-off-precision-and-recall

Trading Off Precision and Recall

image

cancer 여부를 판단하는 알고리즘을 예로 들고 있다. 예를 들어 계산값 판단기준을 0.99 와 같이 높게 잡으면 알고리즘의 정확성은 올라가지만  ( precision 값이 높아짐 ) 많은 경우 이미 암에 걸린 사람도 아닌것으로 판단될 가능성이 있다( recall 값이 낮아짐 ). 반대로 기준값을 0.01과 같이 낮추면 대부분의 사람이 암으로 판단되므로 정확성은 낮아 지고 recall값은 높아지게 된다. 그러므로 적당한 기준값을 설정해야 하며 이는 precision과 recall 값 둘다 적당히 유지 되어야 한다. 두 값을 적절히 조합한 수치가 f1 score이며 이를 알고리즘 성능 판단기준으로 삼는다.

image

기준값을 결정하는 방법으로 F1 score를 이용할수 있다. 


https://www.coursera.org/learn/machine-learning/lecture/XcNcz/data-for-machine-learning

Data For Machine Learning

대부분의 경우에는 많은 양의 training set이 머신러닝의 성능을 높이는데 도움이 된다.그렇지만 꼭 많은 데이터가 있어야만 하는 것은 아니다. 

image

대개 사람이 바로 대답할수 있는 질문의 경우 많은 양의 데이터가 필요하지 않다. 위의 그림의 two 단어 채우기 같은 작업을 들수 있다. 

많은 features를 가지고 있는 linear regression, logistic regression 많은 hidden layer를 가지고 있는 neural network의 경우 많은 데이터가 필요하다.