Bildbearbeitung#
What are images#
Images are made up of tiny pixels, each with a specific color:
Image editing involves changing and manipulating these pixels according to specific criteria.
For this we need arrays, which are a special form of lists (arrays have limited sizes).
Hintergrund wird geladen#
In den Miniwelten können wir den Hintergrund mit der Funktion arr = background.to_colors_array() laden.
Wenn wir z.B. dieses minimale Programm schreiben:
from miniworlds import *
world = World()
arr = world.background.to_colors_array()
print(arr)
world.run()
, dann bearbeiten wir im Folgenden den Standard-Hintergrund:
You receive a nested, two-dimensional array that looks like this:
[[[150 150 150]
[150 150 150]
[150 150 150]
...
[150 150 150]
[150 150 150]
[150 150 150]]
[[150 150 150]
[150 150 150]
[150 150 150]
...
[150 150 150]
[150 150 150]
[150 150 150]]
[[150 150 150]
[150 150 150]
[150 150 150]
...
[150 150 150]
[150 150 150]
[150 150 150]]
...
[[150 150 150]
[150 150 150]
[150 150 150]
...
[150 150 150]
[150 150 150]
[150 150 150]]
[[150 150 150]
[150 150 150]
[150 150 150]
...
[150 150 150]
[150 150 150]
[150 150 150]]
[[150 150 150]
[150 150 150]
[150 150 150]
...
[150 150 150]
[150 150 150]
[150 150 150]]]
The innermost list represents a 3-tuple of colors. [150, 150, 150].
These describe the red, green, and blue components of each pixel. Each color is created by “mixing” these 3 primary colors:
The minimum value for each color is 0, the maximum value is 255.
The image array consists:
From a list of columns
and each of these columns contains a color value for each row (which in turn is a list with 3 values)
Ändern des Hintergrunds#
You can iterate over this array as follows:
from miniworlds import *
grey
world = World()
arr = world.background.to_colors_array()
for x in range(len(arr)): # iterate over all rows
for y in range(len(arr[0])): # iterate over all columns of row
arr[x][y][0] = 0
print(arr)
world.run()
The counter variable x iterates over the columns and selects one column at a time. The counter variable y then iterates over each selected column:
The instruction arr[i][j][0] = 0 sets the first color, i.e., the red component, to 0. The array looks as follows:
[[[ 0 150 150]
[ 0 150 150]
[ 0 150 150]
...
[ 0 150 150]
[ 0 150 150]
[ 0 150 150]]
[[ 0 150 150]
[ 0 150 150]
[ 0 150 150]
...
[ 0 150 150]
[ 0 150 150]
[ 0 150 150]]
...
[[ 0 150 150]
[ 0 150 150]
[ 0 150 150]
...
[ 0 150 150]
[ 0 150 150]
[ 0 150 150]]]
We can now load this array again as a background with the command background.from_array(arr), this is what the complete program looks like:
from miniworlds import *
world = World()
arr = world.background.to_colors_array()
for x in range(len(arr)):
for y in range(len(arr[0])):
arr[x][y][0] = 0
world.background.from_array(arr)
world.run()
…and this is what the result looks like. The color gray loses its red components and thus becomes greenish-blue:
Here we have simply assigned the red value 0 to each pixel. However, we can also assign each pixel a value between 0 and 255.
You can also iterate over this list differently and, for example, color only every second line:
from miniworlds import *
world = World()
arr = world.background.to_colors_array()
for x in range(0, len(arr),2 ):
for y in range(len(arr[0])):
arr[x][y][0] = 0
print(arr)
world.background.from_array(arr)
world.run()
Ebenso ist es möglich, den Wert abhängig von der Zählervariablen i zu verwenden - auf diese Weise kann man Farbübergänge erzeugen, z.B. so :
from miniworlds import *
world = World()
arr = world.background.to_colors_array()
print(arr)
for x in range(len(arr)):
for y in range(len(arr[0])):
arr[x][y][0] = ((x +1 ) / world.width) * 255
world.background.from_array(arr)
world.run()
As the x-value increases, the x-value rises. (x+1) / world.width results in a value between 0 and 1. Multiplying this by 255 gives a value between 0 and 255. If a value is all the way to the left, its red value is minimal. If it is all the way to the right, the red value is maximal.
You receive the following color gradient.
This can also be done with the y variable, and the program can be extended as follows:
from miniworlds import *
world = World()
arr = world.background.to_colors_array()
for x in range(len(arr)):
for y in range(len(arr[0])):
arr[x][y][1] = ((y +1 ) /world.width) * 255
world.background.from_array(arr)
world.run()
At the top, the green value is minimal; at the bottom, it is maximal:
You can now put this together:
from miniworlds import *
world = World()
arr = world.background.to_colors_array()
for x in range(len(arr)):
for y in range(len(arr[0])):
arr[x][y][0] = ((x +1 ) / world.width) * 255
arr[x][y][1] = ((y +1 ) /world.width) * 255
world.background.from_array(arr)
world.run()
You get this color gradient:
Bildbearbeitung#
So far, we have edited a solid color background, but of course, this can also be done with an image as the background.
In this way, we can apply different filters over the image.
We load, for example, the sunflower from above as a background image:
from miniworlds import *
world = World(600,400)
world.add_background("images/sunflower.jpg")
arr = world.background.to_colors_array()
for x in range(len(arr)):
for y in range(len(arr[0])):
arr[x][y][0] = 0
world.background.from_array(arr)
world.run()
As the red values are removed from the image, the image acquires a thorough hue. So we have written a first color filter here. This is what the result looks like:
Next, we manipulate the brightness. To do this, we can multiply the red, green, and blue values by a constant.
from miniworlds import *
world = World(600,400)
world.add_background("images/sunflower.jpg")
arr = world.background.to_colors_array()
constant = 2
for x in range(len(arr)):
for y in range(len(arr[0])):
arr[x][y][0] = arr[x][y][0] * constant
arr[x][y][1] = arr[x][y][1] * constant
arr[x][y][2] = arr[x][y][2] * constant
world.background.from_array(arr)
world.run()
The first attempt looks like this, though!
How did this happen?
Each color value ranges from 0 to 255, but when multiplying, some of our values exceeded 255 and thus “overflowed”. You can recognize this by particularly dark areas that should actually be bright.
So müssen wir sicherstellen, dass das Ergebnis kleiner als 255 ist, z.B. so:
from miniworlds import *
world = World(600,400)
world.add_background("images/sunflower.jpg")
arr = world.background.to_colors_array()
constant = 2
for x in range(len(arr)):
for y in range(len(arr[0])):
arr[x][y][0] = min(arr[x][y][0] * constant, 255)
arr[x][y][1] = min(arr[x][y][1] * constant, 255)
arr[x][y][2] = min(arr[x][y][2] * constant, 255)
world.background.from_array(arr)
world.run()
Bildbearbeitung II (mit Funktionen)#
Brightness#
Oft benötigen wir die Helligkeit eines Pixels. Die einfachste Methode, dies zu berechnen, ist den Durchschnitt der r, g und b-Werte zu berechnen:
from miniworlds import *
world = World(600,400)
world.add_background("images/sunflower.jpg")
arr = world.background.to_colors_array()
def brightness(r, g, b):
return (int(r) + int(g) + int(b)) / 3
print(brightness(arr[10][20]))
world.background.from_array(arr)
world.run()
In the function brightness, the values r, g, and b must first be converted: They are uint8 values, so the result must never exceed 255 (otherwise an “overflow” occurs). Therefore, the variables must be converted to the data type int so that the result of the addition is also an int value and thus can be arbitrarily large.
We can use this to color each pixel gray, depending on its brightness:
from miniworlds import *
world = World(600,400)
world.add_background("images/sunflower.jpg")
arr = world.background.to_colors_array()
def brightness(r, g, b):
return (int(r) + int(g) + int(b)) / 3
for x in range(len(arr)):
for y in range(len(arr[0])):
arr[x][y] = brightness(arr[x][y][0], arr[x][y][1], arr[x][y][2])
world.background.from_array(arr)
world.run()
Colorunterschied und wahrgenommener Colorunterschied#
When working with images and sounds, it is important to know how we humans perceive them.
The perceived brightness does not match the brightness calculated here. However, for the following purposes, the form used here to calculate brightness is sufficient.
Edge-Erkennung#
An important function in image editing is edge detection. This is also important in artificial intelligence because edge detection is a first step in recognizing objects in an image.
How does edge detection work?#
Helper-Funktionen#
To focus on the actual algorithm, we use some helper functions:
Is there a pixel in the image?:
def in_array(arr, x, y):
if x >= 0 and x < len(arr):
if y >= 0 and y < len(arr[0]):
return True
return False
Return all neighboring cells of a pixel:
def neighbour_cells(arr, x, y):
neighbours = []
for x0 in range(x-1, x+1):
for y0 in range(y-1, y+1):
if in_array(arr, x0, y0):
neighbours.append(arr[x0][y0])
return neighbours
We are preparing the image. With
arr.copy(), we can create a copy of the image in which we only store the brightness values.
This way, we don’t have to deal with all three color values later.
This is what the basic framework looks like:
from miniworlds import *
world = World(600,400)
world.add_background("images/sunflower.jpg")
arr = world.background.to_colors_array()
grey_arr = arr.copy()
def brightness(r, g, b):
return (int(r) + int(g) + int(b)) / 3
def in_array(arr, x, y):
if x >= 0 and x < len(arr):
if y >= 0 and y < len(arr[0]):
return True
return False
def neighbour_cells(arr, x, y):
neighbours = []
for x0 in range(x-1, x+1):
for y0 in range(y-1, y+1):
if in_array(arr, x0, y0):
neighbours.append(arr[x0][y0])
return neighbours
for x in range(len(arr)):
for y in range(len(arr[0])):
grey_arr[x][y] = brightness(arr[x][y][0], arr[x][y][1], arr[x][y][2])
world.background.from_array(arr)
world.run()
and now we add the edge detection algorithm:
The algorithm works like this: We calculate the average brightness value of all neighboring cells using the neighbour_cells function and color the image accordingly.
from miniworlds import *
world = World(600,400)
world.add_background("images/sunflower.jpg")
arr = world.background.to_colors_array()
grey_arr = arr.copy()
def brightness(r, g, b):
return (int(r) + int(g) + int(b)) / 3
def in_array(arr, x, y):
if x >= 0 and x < len(arr):
if y >= 0 and y < len(arr[0]):
return True
return False
def neighbour_cells(arr, x, y):
neighbours = []
for x0 in range(x-1, x+1):
for y0 in range(y-1, y+1):
if in_array(arr, x0, y0):
neighbours.append(arr[x0][y0])
return neighbours
for x in range(len(arr)):
for y in range(len(arr[0])):
grey_arr[x][y] = brightness(arr[x][y][0], arr[x][y][1], arr[x][y][2])
for x in range(len(arr)):
for y in range(len(arr[0])):
neighbours = neighbour_cells(grey_arr, x, y)
sum_neighbours = 0
for neighbour in neighbour_cells(grey_arr, x, y):
sum_neighbours += neighbour[0]
mean_neighbours = sum_neighbours / len(neighbours)
diff = grey_arr[x][y][0] - mean_neighbours
arr[x][y] = (diff, diff, diff)
world.background.from_array(arr)
world.run()
The colors are still inverted, i.e., we need a function that inverts the colors again so that the black background becomes white and the edges become black instead of white.
Das geht so:
for x in range(len(arr)):
for y in range(len(arr[0])):
arr[x][y][0] = 255 - arr[x][y][0]
arr[x][y][1] = 255 - arr[x][y][1]
arr[x][y][2] = 255 - arr[x][y][2]
We can still modify and improve this algorithm in various ways. One possibility is to color only cells white that exceed a certain threshold and all other cells black, one could also make the field of neighboring cells larger, …
Try it out!