Carlo Macchiavello

All is possible

Author: carlomacchiavello Page 4 of 18

The proper importance of numbers

By

On 12 August 2021

In Techinque

Numbers are often discussed: sensor Ks, sensor bits, file bitrate, focal length, color depth bits, and so on…

The obsession with numbers is produced by Marketing, which obsessively inculcates in people the idea that a lower number is wrong, is bad, is outdated, or worse inadequate for the modern world… as if marketing is an expert on these numbers (if I had time to waste I would have to have half of every photographic or video brand’s marketing communications corrected for the inaccuracies written, obviously to their benefit).

This sense of inadequacy leads people to think that more is needed, a new product, that the one we own is outdated, causing not only compulsive purchases but an INFINITE production OF ELECTRONIC WASTE that since it cannot be disposed of properly is often pulverized and abandoned who knows where…

I make this point about numbers because I have often addressed these discussions :

  • about the K’s of resolution needed and/or available
  • about color depth
  • on compression
  • on the bitrates of the files

and now we have yet another clash of numbers, as Sony has launched 16bit linear raw sensor outputs against the rest of the world that saves 12-bit logarithmic raw files, excluding Arri cameras that to get around the Red patent of recording compressed raw record in 16bit linear raw uncompressed.

Logarithmic recording is designed to take up less space and at the same time offer the ability to record all the dynamic range information of a sensor or film. So if for Kodak engineers a 10bit log file (Cineon) could be sufficient to record the dynamic range of a film (7-9 stops maximum, remember that cine film is not photographic film, from kodak tables). Perhaps a 12bit log may be sufficient for much more? The same red logs a 12bit log for a camera claimed to have 20 stops of dynamic range.
But leaving aside the factors of … opinions of people (perhaps qualified since they invented all the principles of modern digital cinema) of that type, let’s do some serendipitous math.

8 bits = 256 shades of gray and for each color

10 bits = 1024 shades of gray and for each color

12 bits = 4096 shades of gray and for each color

So even if it were a black and white film we have enough information to cover an 8k with a line of pixels of different brightness vertically, whereas if the film is in color we are talking about 68 billion shades, 719 million, 476736 shades -> 68,719,476,736

AH no…I am mistaken because this figure is relative to the 12bit linear, if we straighten this log curve in a 32bit space (i.e. where it is correct to handle a logarithmic file of such depth) we find that the data corresponds to the 16 bit linear… but taking up much less space.

Now in a world where the digital camera forums are full of people complaining that raw and prores files take up too much space and that they want files in h264 (h265 takes up a quarter, h266 which now has two at the time of writing August 2021 takes up an eighth of the h264), wishing/desiring 16 bit linear recording I find rather amusing.

For those who know me, or follow me, I went the raw route first in stills and then in video as soon as I could because I have always tried to have the maximum workspace in postproduction and so having more data to work with has always been a priority of mine, but that does not mean I am foolish or have endless availability to buy ultra fast media where I can record a linear raw or archive such files, especially since there is historical evidence where Avatar was shot in h264 10 bit 4:2:2 (historical limit of the cameras at that time), spiderman and The Hobbit with Redcode between 5:1 and 3:1, is often chosen because slight compression reduces the harshness of the files and makes them more cinematic.

Personally I would prefer the presence of smarter or better color spaces optimized downward (8bit) than the possibility of recording this type of raw file on low cost machines, because then users who want that type of file, do not know that then post management will be unlikely on many of their machines, and that if they do not have knowledge of a certain type even the advantages in color management will not be appreciable.

Add. I see that on the nice site Cined.com they have published an article with a nice technical explanation of Sony’s 16bit linear by one of our compatriots Francesco Andreola.

Not ones of my usual post

By

On 30 May 2021

In Techinque

Another year has begun, the pandemic is still alive in our nightmares, and … All the posts I wanted to write about how to start 2021 I deleted them.
Day by day, i lived in a loop, I wrote and deleted. ..
the external problems of the world, have vanished in front of those of my world, of my family, losing my mother by a tumor.

Sensitive artists have two choices, someone use pain to create or succumb and are annihilated …
Everyone knows me like a technician, connoisseur of technology and tools of creation, but I was born and live as an artist, a creator of emotions … I have always drawn, illustrated, portraits, images of fantasy with pastels, watercolors, tempera, airbrush up to clay sculpture, to move on to living and moving forms with more or less advanced tech, writer, photographer, filmmaker, animator, and so on and so forth.

My mother was a midwife, but many people have in their homes the oil paintings she made during the long nights.
My mother helped women in the most difficult and complex moment of their life, becoming mothers, and at the same time she used her free time to create abstract or figurative paintings, art was a way for her to release tensions and emotions.

My parents passed on so many values, so many passions, they allowed us to transform a passion into a life's work, me in images and movie, my sister into something much more important, she saves lives, she is an emergency doctor.

For all they have done for us, we have a duty to carry on, hold on, and remember them, past the pain, proud of our achievements, they have always helped us grow and become adults.

I know that now she is with our father, who knew how to illuminate people's lives with his smile, they are happy that they look at us from up there and guide the path of their grandchildren Riccardo and Ginevra with their invisible hand.

We will always make you proud of us.

 

 

File System to be or not to be

By

On 2 December 2020

In Techinque

I’ve been wanting to write two lines about file systems for a long time, both because I see that many problems are caused by the misuse of them, and because I hear so much “rubbish” about media that the time has come to dot the i’s… so the two lines will become a few pages 😀 those who follow me know that I am not one of few words, I hate posts made to rank, I write because I want to be useful and try to be complete.

What is a file system?

When we use a medium, fixed or removable, to write to that medium the operating system must be able to read a “structure” in the medium to organize the data. The file system is used to create and manage this structure. Imagine that the file system is a system of shelves with codes (sectors), which allow us to organize our data so that it is easy to make use of the space and find our data.

When we write a file let us imagine that the contents of the file are a book whose pages are put on shelf 1 (sector 1), if that is not enough some of the pages are put on shelf 2 (sector 2) and so on; when I write the second file he will start from the first empty space he finds nearby, and continue, until the space is full.

When writing, the file system will take care of creating a list of data that will allow us to know, at the request of file 2 in which shelves to retrieve all the pages of book 1, so that when reading and searching the file it will be easy to find all the data that make up the book.

All this is done invisibly to the user. There are dozens of file systems (which those of a computer age like me have known several), although most people end up using only 3-4 file systems in their daily lives :

Exfat – NTFS for those who use Windows
HFS APFS those who use mac and the latter under iOs

File systems are made by those who know, they are foolproof….

Go read up on how a file system error in both deletion and backup management wiped out all of ToyStory 2 in Pixar and then go ahead and come back here terrified…knowing that here it will be your computer that can/will damage your files.

i use x can’t read or write Y

I am sick of hearing nonsense about disks, cards that are not compatible with one or another computer, and out of partisanship they blame the computer brand, when in fact the lack of compatibility is because the brand (which such ignoramuses defend) has decided that their file system is proprietary and therefore not releasing the specifications to have it used properly by other operating systems; both because there are free and paid drivers ($10-12 pretty much what people spend on a gaming app or in a couple of days of wasted ash) to read and write every mayor file system on every operating system.
I have computers with Windows 7 and 10, Linux and MacOs and I read and write from each of them every file system I need, proving that it can be done, I put the links for the respective products to buy, they have no sponsor links, I don’t make any money out of it, but I save my ears from yet another grumbling from the ignorant and lazy people who don’t know how to open Google to search, but always want the ready-made mush … and I can cook well 😛

Why is it important to know what file system I use?

Because the file system determines the strength or fragility of the famous “shelves” where we organize data, and so if we put too many books on fragile shelves everything collapses, causing us to lose data.

Too many people use media superficially and then complain how they lose data, become corrupted, and lose the work they have done. These are the same people who don’t check the sturdiness of shelves or buy a chipboard bookcase and claim to load it with books, sooner or later everything will collapse on them…

Like when my grandmother used to fill her closet with clothes and blankets and complain that the stuff was disappearing–she had broken through the back of the closet by pushing and falling behind the closet. On disks there is no behind, a file that is 80% lost is lost because its structure is damaged.

Knowing which file system we use allows us to know its limitations and frailties, and then use it to the best advantage

But I do video and photos, what do I need it for?

If you ask yourself this question you CERTAINLY need to know.

The majority of camera and camera media use the ExFat file system, developed by Microsoft centuries ago (in 2006) to support flash memories inside the windows CE mobile system, with a thousand fragilities and limitations related to its age..

Using this type of file system it is common that if it is not unmounted correctly Windows when loading the media will say there is an error on the media and propose to correct it NEVER DO IT!!!!!

Sometimes this “fix” process will damage the FAT index (the shelving and contents list), whereas if you do not fix it you will be able to read the data correctly, and then possibly reformat the whole thing.

ExFat (also called FAT64) was created to overcome the limitation of Fat32, Microsoft’s other old File System, which had the limitation of supporting recording files no larger than 4 gigabytes, and in the video realm it is easy to exceed these limits.

Knowing that we are using an old File System with different kinds of limits in structure, especially in data recovery in case of damage, leads us to pay more attention to the media and its management.

how do I avoid problems?

ExFat is simply more fragile than other FSs, and being the most common because of reduced user licenses to pay (not because it is better as many believe) it is good to be careful. Media should be treated with care, whatever file system you use, it only takes a few simple steps to avoid any problems with them.

Removable media for
Cameras, Cameras, Drones, Cine cameras

  • format the media in camera before using it [ALWAYS]
  • insert and remove media with the device turned off [ALWAYS]
  • connect it to Windows and if it asks for repairs DO NOT EXECUTE them but copy the data and then format the media

if you want an extended explanation of why, I wrote an article on good habits on removable media for audio video use.

External hard drives, thumb drives, general media

  • the first time you use it format the disk to the file system you can handle better:
  • Whenever you need to disconnect the disk perform the proper device eject operation from both windows and mac, and wait at least 30 seconds after windows has given permission to disconnect the device, often the system lies and there are still services that write and read from the disk increasing the risk of FAT (File Allocation Table) corruption.
  • If you connect the ExFat disk to a Smart device (smartphones, smart TVs, Smart Boxes), either the device has the unmount procedure (Smartphones and Android tablets) or before disconnecting the media you have to power down the device and wait a few seconds after powering down (some smarttvs turn off the power to the panel, but the system is still finishing powering down when everything seems to be off).

The Windows repair operation starts both when the media is inserted and when the computer starts. If we leave external media plugged in the card reader with the card in it, and reboot the computer, during startup Windows may arbitrarily initiate the chkdsk disk check and repair command, damaging the card structure.

The chkdsk dos command goes to repair the damaged index (damaged according to Windows) but often instead of repairing it it “patches” it, creating more problems in reading files than solutions. Since there is redundancy of indexes, the fact that the main one is damaged

It often happens that such a command is also executed with foreign file systems damaging them, for this reason it might be useful to permanently disable the check at Windows startup, in this way it is executed only if needed and if a media is connected, to avoid unintentional damage from the automatic startup, especially nowadays that Windows decides to reboot for updates even with active software, while rendering post and 3d programs for example….

If you want to disable this operation permanently, you can do it from the Registry

  1. Go to start > run…, type regedit and press the Enter key on your keyboard.
  2. Go to the HKEY_LOCAL_MACHINE\SYSTEM\CURRENTCONTROLSET\CONTROL\Session Manager key
  3. Double-click on the BootExecute key
  4. Transform the value autocheck autochk * to autocheck autochk /k:C /k:E * (where “C” and “E” must correspond to the drive letters of the disks for which you want to disable automatic CHKDSK).
  5. Reboot the computer

You have reached this point in the article and are among those commenting:

  1. This has never happened to me
  2. I don’t have time to waste…
  3. i don’t believe you, the people who develop and make the products know what they are doing.

My most mundane responses are :

  1. Best wishes, may it never happen to you
  2. You don’t have time to wait for the 30 seconds?
    go to facebook, then after half an hour you will realize that the disassembly happened 29 minutes and 30 seconds ago 😛
  3. you are right not to believe me:
    1. you don’t have to believe me.
    2. you don’t have to believe the popup that appears during installation or the first boot of all operating systems that says the product is “as is” and they are not responsible for any damage to third parties.
    3. you don’t have to believe the directions to take out and put in cards with devices turned off that are in every room, camera, TV etc.
    4. you must not believe the disclaimer in every instruction manual that they are not responsible for the contents.
    5. you must not believe the disclaimer in the card or hardisk packaging that they are not responsible for any loss of data and in case of accidental product damage at most they will give you back a new media…

I am actually writing this article because I know too many people who have lost or risked losing data because of this kind of attitude, and I have often helped them with data recovery.

My best wish is that you are always in the first situation, the one where you will never lose data or see your data damaged.

The habit of error

By

On 17 November 2020

In Techinque

An amusing fact that happened today reminded me of an incident that happened a decade ago: in the car with a person, as a newly licensed driver I notice that he did not proceed in the correct manner, the person in question points out to me that I am newly licensed while he has been driving for over twenty years….

The mistake repeated for twenty years gives him the arrogance to claim that he is the one who is in the right…. And I’m talking about a triviality like giving right of way on zebras….

In another situation, discussing with a certified xx technician, multi qualified yy in xx categories he states that it is impossible for a certain Windows problem to exist, because he in his 6 year career has never seen it [an error that Microsoft instead acknowledged and fixed for me].

Very often those who are familiar with a topic can make the mistake of not questioning the correctness of the knowledge.
Personal experience is important, but no one has lived through everyone’s life.

I know many artists in many fields, who while making mistakes achieve miraculous results, and since they have been doing it for over 30 years they are convinced that it is the best method (I do not use the correct term because there could be many nuances), not questioning the fact that they could take less time, less effort, or have more flexibility…

after all, of Leonardo da Vinci only one was born….

How many times in our lives, and I question myself first, has security in our own knowledge, in our own experience, led us to Absolutism?

I for one made this mistake often, then I passed the age of 25 and realized that the world is an infinity in the making, and it is not possible to have the certainties of anything, in any area or situation.

If we think about medicine, from one decade to the next, certain treatments were abolished, because they brought more harm than benefit; think about construction, we used carcinogenic products such as asbestos for decades; think about physics, where you thought you had discovered the smallest element, but thanks to different methods, more sophisticated instruments, or new technologies every few years you discover a smaller element inside.

If we think about the image, until a couple of years ago or so we thought that a bad image could just be thrown away, whereas today thanks to the magic of AI we can refocus blurry images, zoom in by regenerating the details of very small images, intelligently interpolate frames to make ralenties smoother, restore damaged images thanks to neural networks that intelligently and consistently regenerate details, and so on…

I conclude by quoting from the message of a dear friend, who asking me the technically correct for exposure, I had to disappoint him by not knowing how to give him a unique answer, because the tools are only references …

The classic 18% gray card is used to have an exposure reference, an average value to expose the generic human skin, but already going from north to south Italy the skin can be lighter or darker.

Depending on the sensors, the dynamic range, and how it is distributed in the different ISOs, it is convenient to expose one way or the other, to experience the DoP on the camera (as once upon a time exposure tests were done on batches of film taken for film shooting).

Dogmas exist only in religion, in all other areas everything changes, there may be more than we know, more than we can know, or there will be more, which is why constant growth and change is important.

I myself in this blog, as I have learned, as I have discovered my mistakes I have aligned and corrected articles to avoid the propagation of incorrect information.

Mattebox serve?

By

On 3 October 2020

In Techinque

I have noticed that the authoritative Redshark site, of which I am a regular reader, often features topics I discussed in posts two or three years earlier; I find it flattering to have anticipated them by so much time.

Today I am going to do the opposite, which is to use the cue from one of their articles to answer a frequently encountered question: do you really need the Matte box?

What is it and what is a Mattebox for?

The good gives a list of reasons to which I refer and add my own thoughts.

  1. Look cooler
  2. Protect lenses from side light and avoid unwanted flare
  3. Mount filters.

and I would add

  1. Disciplining the operator in how to prepare the room
  2. Study the light before pressing rec
Look cooler (when size matters).

Potato Jet shows in this video how it is not the rig that makes the kinematic image
ps. follow the link, he is an interesting youtuber

For some (psychological) reason there are elements that create mental associations in people’s minds, the Mattebox is one of those elements, Mattebox equals cinema, professionalism.
So any camera with the Mattebox becomes professional in the eye of the less experienced, or the customer.
In many areas the digital revolution has been a disadvantage, because customers accustomed to associating camera size with the professionalism of the operator (larger size = economic investment = shooting quality) find themselves disgruntled at seeing increasingly compact machines.

One well-known brand in the mid-2000s had a series of cameras dedicated to wedding videographers and those making corporate and industrial films, i.e., the exact same electronics, optics, controls and sensors as the more compact, but professional cameras in a more engine broadcast “outfit” for those who needed to show clients the size of their “professionalism.”

By the mid-2010s the trend had reversed, you could show up with good cameras, film cameras, but if you didn’t have the trendy VDSLR … it looked like you had a toy on your hands … the world is about appearance, not substance, in most cases.

 

 

Page 4 of 18

Previous

Next

Powered by WordPress & Theme by Anders Norén

error: Content is protected !!