Skip to main content

Posts about programming (old posts, page 1)

Study, Day 6

Did a little bit of french and geography.

MITx, 6.00.1x, Introduction to Computer Science and Programming Using Python

Doing compsci but no notes today, because i am busy learning.
Am going to have to revisit oop sometime again.
will make notes then

But to summarise,

  • i learnt oop
  • i learnt about generators,
    • which to me sounds like a tap on a loop.
    • instead of a bucket of water, i get it by the mug.
    • here’s a mug. here’s another mug. here’s another mug.
    • enough? then i can turn the spigot off
    • no need to fill the whole bucket (run through a whole dictionary, or carry out a huge operation) when I can just take however much I need.
  • i learnt about o notation to figure out how long code takes to run, if want to later optimise stuff
  • sorting lists with sorting algorithms
    • bubble sort
    • selection sort
    • merge sort efficient
  • i learnt how to plot using python

Have finished all the videos and will now move on to the problem sets.
I had so much fun, I actually was sad to watch Prof Grimson’s last video.
The course has given me a really good grounding on the basics.
And should help me start my journey.
While I understood the what, MIT 6.00.1x gave me a good grounding on the how and the why.

Study, Day 5

Did some geography for 12th in the morn.
Slow, steady, iterative progress there.


MITx, 6.00.1x, Introduction to Computer Science and Programming Using Python

exceptions & assertions

Exceptions

  • What happens when program execution comes across something that it did not expect?
    • like you could go beyond the bounds of a list
    • i try to reference a variable that isn’t there
    • i mix data types. (i dun dis :P 'a'/9)
  • This is basically Python telling me, ‘I don’t get this! Whatcha talkin’ bout Willis?’
  • Python has a whole host of errors, the names of kinda give me clues to what it thinks the problem is. There are
    • SyntaxError (I boo booed typing somewhere, Python does not ‘get’ me)
    • NameError (Can’t find a variable. Did I put it in? Did I call a module correctly?)
    • AttributeError (Errors with classes. Am I calling them wrong?)
    • TypeError (Am I doing string operations on numbers? Or vice versa?)
    • ValueError (Something is wrong with my results. Is it in a form the program understands?)
    • IOError (something that happens when i make boo boos with files)
What do I do with exceptions?
  • Fail silently (like fart silently and then look about innocently)
    • bad idea, though, really bad idea :)
  • Return a “error” value
    • make sure that you have code that can check for and handle the values you choose
    • makes code a little clunkier
  • Just stop execution, and signal an error condition
    • in Python that’s called raising an exception like so,
      raise Exception("Good God! You’re dividing letters with numbers?!")
Dealing with Exceptions. How do I handle them?
  • Use the try & except keywords.
  • As in you’re telling Python, ‘try’ this code. If everything works well, great!
  • But if there’s an error, jump to the ‘except’ clause, and execute what is in there
  • And then carry on with the rest of the program
  • I can then get fancy. I can have multiple except clauses.
    • one to handle typos
    • one to see if my input is valid
    • one to handle division isssue
    • and a final one as a catch all, if all else fails.
  • There is also an else that i can pair with a try
    • the else will execute code if try block it is associated works, with out any exceptions.
    • like ok, try this. all worked? then go that else block and do that thing and then carry on with the rest of the program
    • to my mind it kind of is the main program, because this is the part that always executes.
    • so as I iterate, i keep removing things from try and move them down to the mandatory else clause
  • And then there is finally
    • executed after try, except and else
    • even if they raise errors or cause a break
    • useful for cleaning up after a mess :P (close a file for e.g.)
    • like ok, here’s a booboo, but I better put everything back in it's place and undo stuff if I can
    • like do no harm.
Exceptions as control flow
  • turn the idea of exceptions on its head
  • instead of waiting for errors and catching them and then raising exceptions
  • we could check ourselves for expected values and raise our own exceptions if results don’t match expectations :)
    • like if i am unable to produce a result that is consistent with the specifications of a function
  • to do that we use raise
  • we raise exceptions
    • like raise iAmDisappointedinYouError("You should have written this better")
    • or more likely you want to raise an existing known error like raise TypeError("You need to enter a number! For the love of God, enter a number!")

Assertions

We can take the idea of proactive exception raising, to another level.
We can defensively program so that we prevent situations that lead to unexpected results.
This idea is what we call an Assertion

  • Remember those docstrings we typed where we expected a certain of inputs and guaranteed outputs?
  • you can think of assertions as the bouncers who enforce them
    • i assert that this should be true. if not, raise an exception
  • you can use the assert statement that in turn will raise an AssertionError if conditions aren’t met
    • could be on inputs (typically, mostly used here)
    • on an intermediate stage of the program
    • if i want to be really careful, i can use it to check output too
  • having these checks makes it a lot easier to locate the source of a bug
    • because if an assert fails, the bug most probably should be just before we called this function
  • Where should you use assertions?
    • so spot bugs as soon as they are introduced and clarify where they happen
    • as a supplement to testing
    • filter and check input
      • check types of arguments or values
      • check that invariants on data structures are met
      • constraints on return values are met
      • violation of constraints on procedure (no duplicates in a list, for e.g.)

Study, Day 4

Back at it after a short personal break. Lots of stuff to catch up on!

MITx, 6.00.1x, Introduction to Computer Science and Programming Using Python

Learning about lists.

  • They are
  • iterable, i can run through its elements one by one and do stuff
  • mutable, i can change things inside them
  • append, will append stuff to the end
  • extend, will take another iterable (another list, for example) and then take those elements one by one, and tack them on the the end of this one
  • i can go back and forth, between lists and strings, converting one to the other and vice versa

  • i can sort and reverse sort the items in them.

  • sorted will return a new list
  • .sort will mutate and sort the list itself as will .reverse.
  • since i can change them at will, i need to be careful about things that might break.
  • 10 names pointing to the same list. changing something in one, changes them all.
  • consider cloning a list and making seperate copies. (this is how’d i’d naturally do it. but is it expensive with resource usage?)
  • one example is iterating over lists
    • if you modify them as you run over them, it can lead to all sorts of trouble and confusion, if items get added, removed or changed
    • best to clone the list, iterate over the clone and modify your original
  • i can nest lists. have lists of lists!

  • you’ll get confused and make mistakes. it’s ok.

Learnt about Functions as objects

do i still remember functions?

  • pieces of code (functionality, my customised shit) that i write that i can call from elsewhere.
  • first class objects in python
  • can be elements of data structures (i can have a function in a list)
  • can appear in expressions
  • i can assign a function to something (a variable for example)
  • can use it as an argument to another function.
    • (i have a function that does things to lists. this function could call a function that sorts and sanitises the list, first)
  • basically, i can use them the same way i’d use numbers, or strings or lists

this inceptive programming, using function as arguments to others and combining them with other elements like lists (or inside lists to do shady shit on other lists)? it’s fancily called higher order programming.

  • i could apply a series of functions to a list
  • i could apply a series of lists to a function

and all this stuff generalised and abstracted are called higher order procedures
python provides something like this, called map
very simply put, map

  • takes a function that expects a single argument. (len for example or abs)
  • and then runs them on a list (or it creates a list where it applies that function to each element in it.
  • what it returns is an iterable (think of it as the elements of the list, coming out one by one, but you get your grubby paws on them, before they go into some defined data structure like a list.)
  • you can then do what you want with those hot cross buns, er, elements. print them. put them in a list. whatever. it’s all good.
  • i can now amp this up to a 11. map can
    • take n functions and run them on n collections of arguments.
    • simple example, run through the elements of two lists, compare them, and pop out the lower number for each element. print or save or whatever.
    • mind fucking blown. insert Keanu going, ’I know Kung Fu!’

Common operations across strings, tuples, ranges and lists

if i assume seq is any of these data structures, i can,

  • seq[i] - get the ith element in that list or string or range or tuple
  • len(seq) - get its length
  • seq1 + seq2 - concatenate two of them (not range)
  • n*seq - repeat them (not range)
  • seq[start:end] - slice and dice them
  • e in seq - will return true if e exists in the sequence
  • e not in seq - will return true if it’s not
  • for e in seq - will iterate over each element in that sequence

Dictionaries

Why? They save me time!
Instead of indexing on numbers, like I do with the 1st element or the 0th element of a list, i can just index on things, I define myself

  • Like there’s a list of names and I can then attach attributes to them. relationships, address, phone number, whatever.
  • works just like a real dictionary. i can look up a word and then see all the information related to that word.

In python you create dictionaries using braces. and store data using key value pairs. here’s a few

  • dictionary_of_names = {} creates an empty dictionary.
  • dictionary_of_ages = {'Jason':40, 'Tess':48; 'Leo':76} creates a dictionary of names (keys) with their related ages (values)
  • and now I can just ask for Tess’s age with a dictionary_of_ages['Tess'] without having to lookup where and at what location Tess is in the dictionary

  • dictionaries are mutable

  • I can add entries (dictionary_of_ages['Puppy'] = 4 will add the age of my stray doggo to the dictionary)
  • I can test for existence of entries ('Leo' in dictionary_of_ages)
  • I can delete entries with a del
  • i can list my keys by calling the dictionary like a function using the keys method like so (dictionary_of_ages.keys()) and similarly I can get at the values with a dictionary_name.values()
  • unlike a proper dictionary though, there is no order to the way things are stored in a software dictionary. mostly its the order you shoved stuff in; and that can change.
  • on keys and values
  • values
    • can be any type
    • can be duplicated (different people having the same age)
    • can actually be any data structure (lists, numbers, strings, other dictionaries)
  • keys, on the other hand
    • have to be unique (can't have two folks with the same name. if they do, then probably time to think of indexing on something else?)
    • need to be immutable (which restricts me to ints, floats, strings, tuples or booleans. can’t have strings in there)
      • actually according to Prof Grimson they need to be hashables, but I’ll cross that bridge when I come to it.
    • careful using floats as keys, because of accuracy issues.

So because of their flexibility on what they can store and index on, dictionaries are much more capable than other data types

Testing & Debugging

It’s like soup!
You’re making soup and there are bugs falling in, from the ceiling.
So how do you get good soup?
You could,

  • Check the soup and remove the crawlies if any. (that is testing your code)
  • Keep the soup pot covered. (defensive programming)
  • Clean the kitchen, so there are no bugs (eliminate the source of bugs. debug it. kill it.)

Defensive programming is,

  • how do you structure your code?
  • how do you write code that plans ahead?
  • it has three parts
    1. write specifications for the functions you write
      • use docstrings
      • mention what you expect as input and what you will provide in return
    2. Write modular programs
      • break up your program into obvious pieces.
      • something that collects data for e.g., and something else that works on it (could be one, or more pieces) and something elser that delivers it out.
      • help you while testing, because you can test each little piece seperately
    3. Check conditions on inputs and outputs.
      • i don’t know what this entails, will update this later

Testing code, basically boils down to checking inputs and outputs.

  • What did i expect to come in? What is my expected result?
  • If stuff’s not working, what do I do to debug it?
  • What can I do to stress test and break my program?

Debugging
Yikes! I have a bug!
How do I kill it? What do I want to do to fix this?

  • Look at the events that led to the error
  • Ask yourself, “Why is it not working? What’s causing that?”
  • and then having found that, “How do I fix it, so all is right with the world again?”

Set yourself up, for easy testing and debugging

  • Design code so this part is easy!
  • Is your code modular?
  • Are docstrings in place? What are you expected inputs? Output?
  • Document your assumptions. Why did you write this piece of code the way you did?

When you test you can,

  1. Unit Test
    • validate each piece of your program
    • test each function seperately
  2. Regression Test
    • if you find bugs, test for them the next time around
    • catch reintroduced errors that were previously fixed
  3. Integration Test
    • Does the overall program work?
      It kind of feeds each other like a cycle, so don’t hesitate to go around and round the testing circle.

Testing Approaches

  • Intuition
    • what do you think you naturally need to test?
    • what makes sense?
    • just pick things at random
  • Black box testing
    • use you program like an appliance
    • it says it does this on the box. well does it do it? does it do it well?
  • Glass box testing
    • look at the code and then test each path in the code
    • can’t do this with every path properly sometimes
    • do the big branches

Bugs

  • Overt
    • the program crashes
    • your computer hangs
  • Covert
    • the answers might not be expected
    • they might look ok, but are actually wrong. (off by 1?)
  • Persistent
    • happens every time you run the program
  • Intermittent
    • happens only some times


  • Overt & Persistent
    • Easy to detect
    • use defensive programming to try to steer bugs to fall into this category
  • Overt & Intermittent
    • if you can find what’s causing it, yay!
  • Covert

    • Really dangerous. you might be relying on wrong results and data that you wrongly trust!
    • could be a really long time before you catch something like this
  • Be Patient. this will take time to get good at

  • Use print statements liberally
    • when?
      • enter function
      • parameters
      • function results
    • use the bisection method
      • divide and conquer!
      • print halfway and then decide which half to focus on
  • Be systematic. Use your little grey cells.
  • Figure out the common errors (type errors, value errors, syntax errors)
  • Logic errors, the program does not work as expected
    • think of programming as a novel you write
    • dream up the path
    • explain your code
    • walk around trying to create the model in your head
    • how did i reach this place? this bug?
    • try explaining it to some one (or your rubber ducky)
  • Use the scientific method.
    • look at what you have,
    • figure out what could be causing the error,
    • think about what could fix it,
    • and then try it.
    • is it fixed? no? go back and try again :)

Work with small things.
Work in increments.
Test and debug it.
Use backups.
Have versions.
Test and compare across versions.
Feel free to work up and down the version tree

Professor Grimson is awesome, and teaches me the way, I imagined some one teaching me CS basics. So many aha moments!


Study, Day 3

Morning

Lets see what this day holds :)

French

  • Did a review session
  • Made 15 flashcards

MITx, 6.00.1x, Introduction to Computer Science and Programming Using Python

  • goofed off on twitter and read articles on blogs instead of studying
  • learnt about tuples.
  • it’s been an hour & a half and I am still terribly distracted.
  • doing the dishes to regain focus. be back in 30.

Nope. cannot focus.
Annnnnd now there is no power. This day is done!
Will just sit at the window and enjoy the rain :)
Hopefully tomorrow is a better day.


Study, Day 2

Just did the MIT course all day today, because my next problem set is due tonight and I did not want to fall behind again.
Got distracted a bit, but a lot less than yesterday with the way Nikola renders headers, and youtube videos of sausage making :P
Now that I am going through the class, I realise programming is not what I imagined.
It is at once, much simpler and a bit more complex than I thought it to be.
More than that, I realise I can do this. :)

The day isn’t over yet. Will attempt to solve the problem set at night.

French

  • Did a review session
  • Made 25 flashcards

MITx, 6.00.1x, Introduction to Computer Science and Programming Using Python

  • Good programming
    • More code is not necessarily a good thing
    • A good measure is the amount of functionality, the utility, your program provides
  • The notion of abstraction
    • like a thing you know how to use.
    • the details of how it is internally are irrelevant to you
    • the box provides utility and you know how to use the controls to get what you want. that is all that matters.
    • like a phone. or a tv. or a scooter.
  • The notion of decomposition
    • i can take all those dumb black boxes that I know how to use
    • and mcgyver them together to do something I want.
    • use independent pieces of codey things that each do something and then glue them all together to get the program I want
    • or to invert, can i break my big problem down into small independent pieces that I can write, or use other folks’, that I can stitch into my own little frankenstein?
  • Decomposable Code, divide code into modules that
    • are self contained
    • are reusable
    • keeps you and the code organised
    • keeps things coherent
    • can be done with functions or classes
  • Abstractable Code, build a black box that
    • cannot see details
    • does not need to see details
    • does not want to see details
    • hides the gory details from you. or just the fact that your magical thing is made of small boring pieces :)
    • can be done with function specifications or docstrings
  • One new aspect of functions I learnt was that they have specifications.
    • Thinking back, it’s a well, duh! point, but I’m glad I learnt it explicitly.
    • there’s a contract between me, the author of the function, and its users
    • there’s a set of assumptions that the function has. this is what I expect, this is what I want, these are the values you need to pass, this is what the environment needs to look like, what is the phase of the moon?, yadda, yadda
    • and if those assumptions are met, there are guarantees I can make about the output.
      • pass these numbers in, in this sequence, and you will always get this output. put the dough in the oven, when the new moon is rising, and you will always get delicious pizza
    • in python, a docstring documents what the function assumes and guarantees. good programmers do this to help other folk using the function and to save their own future dumb selves
  • I learnt recursion
    • from what i understand, this means that i take something i know and then use it to repeatedly tackle a complicated problem, if said problem, lends itself to being broken down that way.
    • break it down until i reach a step where i know i can do all of the operations on my own.
    • multiplication is a good example.
      • I know that if when a*b when b is 1 is just a.
      • so I can write a function that keeps adding a to itself while decrementing b until b reaches one and et voilà, that value of a is my answer
      • I got that backwards. I know that i need to repeatedly add until b is 1. so i just keep adding a to a function that just asks if the value of b is 1. if not just add a to that same function where the else states that I reduce b by 1. when b is 1, i just return a which will be the first of the added as and the function then begins looping outwards and backwards adding a. I realise I have horribly explained it, but it’s somehow more intuitive and more elegant to my mind and fun to watch, so I’ll let Professor Grimson do it much better than I can
      • I still don’t totally get it, but I get that this is cool and makes tackling hard problems easier. Hopfully more understanding will come with time.

P.S. A note to student planet readers, if you miss some posts in that feed, check the site to see if I wrote anything (or manually subscribe to the main feed.) I might be uncomfortable pushing certain language or frustrations of mine to other learners at large.


Study, Day 1

Day 1 is an disaster of fantastical proportions.
I did lots of stuff.
But I got nothing of consequence done.
Save one big thing.
I managed to buckle down and study for six hours.
That counts as a win, a big one in my book.
Will hopefully get more done tomorrow.
If you are following this blog, the days might soon seem out of sync.
That’s because I’ve decided that Wednesday, Thursday and Friday are my days of study. Monday & Tuesday are for work, while Satuday and Sunday are for home and family.
I need to practice what I preach and have margins and boundaries in my life.

French

  • Created cards and a review sesion

Python Problem

  • I decide to tackle this first thing in the morn and sank like a stone.
  • I yak shaved for 2 hours (git issues, setting up a dedicated desktop space for study, rsync issues, syncthing issues, crontab issues, more git issues)and then took nearly another hour trying to remember things and then by the time I had some idea of how to tackle the issue, it was lunch time. Hopefully better luck tomorrow

The MITx, 6.00.1x, Introduction to Computer Science and Programming Using Python course

  • I am hopelessly behind and playing catch up.
  • Already missed the deadline for the week one exercises.
  • Hopefully will be all caught up by this time next week.

I learnt

  • that iterables are things that can be counted. like beads in a string, or on a rosary.
  • strings are a form of iterable.
  • Bisection searches to find square roots are much more efficient than having to go slowly guessing our way up.
  • I should not compare floating point numbers (e.g. test for equality). Their internal representations might just be subtly different. Instead use the absolute value of the difference of the two floats. abs(x-y) instead of x==y
  • Start with a basic set of code, check to see what it runs on, and then see if small changes to the code can solve other similar problems or improve the efficacy of existing ones. (Newton-Rhapson better to find roots than Bisection guess better than incremental exhaustive guessing)