Strange rotation in C# graphics

I am using OnPaint method in my class Class1 : Panel.

protected override void OnPaint(PaintEventArgs e)
{
    base.OnPaint(e);
    Graphics g = e.Graphics; 
}

to rotate and draw rectangle I am using

Matrix m = new Matrix();
m.RotateAt(90, rotationPoint);
g.Transform = m;
g.FillRectangle(Brushes.Black, rectangle)

the problem is, that rotation isn't working as I want it to.

Red square is rotation point and it's located in the middle-top of rectangle. How to set x, y and rotation point so rotation would work properly?

After rotating at 90 degress it should look like this

red pixel is still at the same location.

Rotation point is not the point, which you want to rotate. It is point, around which graphics is rotated. So if you draw a rectangle on the top of the graphics and want to rotate it (rectangle) - then you should set rotation point as center of graphics and rotate image to 90 degrees.
Here is example, that does almost what you want:

base.OnPaint(e);

var g = e.Graphics;
var width = g.VisibleClipBounds.Width;
var height = g.VisibleClipBounds.Height;
var rotationPoint = new PointF(width / 2, height / 2); ;

// draw center point
g.FillRectangle(Brushes.Red, new RectangleF(rotationPoint.X - 5, rotationPoint.Y - 5, 10, 10));

using (var path = new GraphicsPath())
{
    var rectangle = new RectangleF((width - 10) / 2, 0, 10, 10);
    var m = new Matrix();
    m.RotateAt(90, rotationPoint);
    path.AddRectangle(rectangle);
    path.Transform(m);

    // draw rotated point
    g.FillPath(Brushes.Black, path);
}