Tuesday, December 28, 2010

Activation/Deactivation of python virtualenv upon entering a directory

It's not a new or original idea – I've heard about it from Dmitry Gladkov but as usual didn't remember details. So, I've created my own implementation of activation/deactivation of python virtualenv:

#!/bin/bash

PREVPWD=`pwd`
PREVENV_PATH=
PREV_PS1=
PREV_PATH=

handle_virtualenv(){
  if [ "$PWD" != "$PREVPWD" ]; then
    PREVPWD="$PWD";
    if [ -n "$PREVENV_PATH" ]; then
      if [ "`echo "$PWD" | grep -c $PREVENV_PATH`" = "0"  ]; then
         source $PREVENV_PATH/.venv
         echo "> Virtualenv `basename $VIRTUALENV_PATH` deactivated"
         PS1=$PREV_PS1
         PATH=$PREV_PATH
         PREVENV_PATH=
      fi
    fi
    # activate virtualenv dynamically
    if [ -e "$PWD/.venv" ] && [ "$PWD" != "$PREVENV_PATH" ]; then
      PREV_PS1="$PS1"
      PREV_PATH="$PATH"
      PREVENV_PATH="$PWD"
      source $PWD/.venv
      source $VIRTUALENV_PATH/bin/activate
      echo "> Virtualenv `basename $VIRTUALENV_PATH` activated"
    fi
  fi
}

export PROMPT_COMMAND=handle_virtualenv
Just paste this code into your
$HOME/.bash_profile
and place
.venv
file with declaration like below:
VIRTUALENV_PATH=$HOME/.envs/sampleenvironment
And it should works like a charm. Script only for bash!

Thursday, December 23, 2010

Django model migrations with South: how-to migrate ForeignKey relation to ManyToMany

Yesterday I had an non-trivial for newbie South user issue: I need to migrate a big database table with existing FK relations to M2M relations and keep in safety all of existing data during migration.

So, I hadn't figure out in documentation how I can do it without handmade migration code in forward method. Below I described how I've fixed it:

1. Here is my initial model (sample), my application called app1
class TestData(models.Model):
    field1 = models.CharField(max_length=200)
    field2 = models.CharField(max_length=200)


class Knight(models.Model):
    name = models.CharField(max_length=100)
    additional_field_new_name = models.CharField(\
        max_length=155, default='')
    data = models.ForeignKey(TestData)

2. I've added new column data_new to Knight model with M2M relation to TestData:
data_new = models.ManyToManyField(TestData, \
        related_name='testdata_info')

3. Start schema migration:
python manage.py schemamigration app1 --auto

4. Change forward method in our new migration, code after creation of new column/proxy model:
for obj in orm.Knight.objects.all():
    obj.data_new.add(obj.data)
    obj.save()

5. Make migration:
python manage.py migrate app1

6. Remove column data from model and make migration; rename m2m column data_new to data and start schema migration
# remove column "data" from model
python manage.py schemamigration app1 --auto
python manage.py migrate app1

# rename m2m column from "data_new" to "data"
python manage.py schemamigration app1 --auto

7. At this point I need to keep all existing data in proxy table but South won't do that for me. So, I need to change forward method of migration (actually I need to replace generated code to following):
db.delete_unique('app1_knight_data_new', ['knight_id', 'testdata_id'])

db.rename_table('app1_knight_data_new', 'app1_knight_data')
db.create_unique('app1_knight_data', ['knight_id', 'testdata_id'])

Update: for Postgres (and I assume for MySQL with InnoDB db storage too) order of above commands is important: you have to delete old unique constraint before renaming of the proxy table.

8. After that just make migration:
python manage.py migrate app1

So, we've done and everything in safety.

Friday, December 17, 2010

Keep synced production media files with your development environment- the "dirty" way

As I wrote in previous post, it's really problem personally for me to keep synced media between dev and production environment. I don't examine cases when on production available really a lot of data.


So,
Copy user-related media files – usually solved via ignore files, symlinks etc. but there is NO STANDARD or SOLUTION - Unsolved
Quick and dirty solution:
# 1) go to parent folder of your 
# static_media (usually project's root folder)

# 2) execute
rsync -avzu -e ssh mylogin@remotehost:~/path/to/static_media ./

 And execute it every time you need have synced media files with your remote host. 


Update:
 I've hacked previous version of script to exclude files which already in git repo.
rsync -avzu echo `git ls-files | \
   grep static/ | \
   xargs -I file echo --exclude="file"` \
   -e ssh mylogin@remotehost:~/path/to/static_media ./

Wednesday, November 17, 2010

Django deployment problem

Actually, there are several problems:

  1. Upload to the server - usually solved via SCM like git, mercurial, subversion etc – Solved
  2. Copy application files to proper destinations, reboot application - bash/fabric etc. – Solved
  3. Copy user-related media files – usually solved via ignore files, symlinks etc. but there is NO STANDARD or SOLUTION - Unsolved
  4. Migrate database, maybe it's possible to solve it via some migration tool like South but actually I don't know and at this point it's still - Unsolved
  5. Fallback to previous version (=any version?) – unsolved and almost impossible to keep data in actual state trough migrations - Unsolved 

So, what I think about it.


Database Migrations
I guess it's possible to solve via South. Another way is to write my own migrations with specific format - SQL with bash/python etc. I will dedicate some time to learn more about migration.

Copy user-related media files
I saw something about it in latest version of django. But sometimes I have to work with old releases (1.1.1) of django I need to keep it independent from django - I'll create configuration file which will consist from list of directories where User's data will be stored. Then some playing with symlinks and it will be solved.

And, at least
Fallback to previous version
There is no rocket science to change symlinks. But I still can't decide what I can do with database at this point.

How to handle variations with Javascript within Satchmo's product template

My goal was to list a set of variations on product page and redirect user to related product page if variation was changed. For example we have two variations:

  1. Size
  2. Colour
And values of variatons:
  1. Colour: red, black, white
  2. Size: M (middle), L (large), S (small)
So, first of all I display this variations on product page and when user change something (colour or size) then redirect to product with selected colour and size respectively.

1. Show dropdowns with variations:
{% for option_group in options %}

{% endfor %}

2. Store all possible variations for javascript to find related product if some of variations was changed on the page:
// list of available variations of current product
    var variations = [
    {% for product_variation in product.configurableproduct.productvariation_set.all %}{
        'url': '{{ product_variation.product.get_absolute_url }}',
        {% for option in product_variation.options.all %}
        'group-{{ option.option_group.id }}': '{{ option.value }}'{% if not forloop.last %},{% endif %}
        {% endfor %}
    }{% if not forloop.last %},{% endif %}{% endfor %}];

3. Add JavaScript code which will handle variations changes:
  /**
   * Select & go to variation which have been selected via select box.
   * This functionality require a set of available variations of the product
   * which consost from something like that: [
   * {'url': 'url_to_product', 'group-': '',..}
   * ]
   */
  var any = function(arr){
       for(var i = 0; i < arr.length; i++){
           if(arr[i] != true){
               return false;
           }
       }
       return true;
  }
  $('select.priced').bind('change', function(){
      var groups = {};
      $('select.priced').each(function(){
          if($(':selected', $(this))){
              groups[$(this).attr('id')] = $(this).val();
          }
      });

      $(variations).each(function(i, vr){
          var flags = [];
          
          for(var grp in groups){
              if(vr[grp] == groups[grp]){
                  flags.push(true)
              }else{
                  flags.push(false);
              }
          }
          
          if(any(flags)){
              location = vr['url'];
          }
      });
  });